The field of engineering requires quality assurance. Whatever their discipline, engineers have to create and build often complicated products. Sometimes, they review their designs and the end results themselves in order to make sure everything is functioning as it should; in other cases, that work is a separate job performed specifically by a quality assurance engineer.
Learning experience platforms are no exception, which means that Extension Engine’s team of 39 software engineers and quality assurance engineers in Split, Croatia, faces a huge job. Using analytical thinking, critical thinking, and creativity, these teams work together to shepherd projects through their life cycles. They’re led by Director of Engineering Miroslav Dojkic, who oversees the teams, as well as the team leads who manage the day-to-day operations of each job and advise individual team members on their career paths.
“We mentor and teach other engineers. The meaty part of our job is summoning the software demons to help us meet deadlines,” Miro jokes before explaining that there’s a lot more to an engineer’s job at Extension Engine than coding. “[The first part] is engineering the project, then engineering the process, and last, engineering the organization.”
Engineering the Project: Project Life Cycles
Miro’s teams’ projects follow the cycle of plan, build, ship. The first step is the discovery phase. “This is where we try to identify the business goals for the client and try to find a way to support [the goals],” he explains. “The engineer’s role in all that is first and foremost to determine the feasibility of our solutions and to give ballpark estimates so that we actually know we can achieve something in the given timeline.” Although it’s hard to accurately make these estimates, they’re helpful in giving everyone on the project something to aim for..jpg?width=960&height=540&name=Copy%20of%20Extension%20Engine%20Engineering%20Department%20(1).jpg)
Next, the teams figure out which technologies they can use to support the goals they’ve identified and decide how to organize the project to allow the engineers to “develop functionality” for it. They also lay out the software development life cycle so that the team knows what goals they’re working toward regarding shipping dates for the software. “After we are done with the start,” Miro says, “we can take a look at the rest of the life cycle. Requirements do change over time, and we learn a lot when we put our ideas against reality.” At this point the team discusses risk mitigation and makes plans for facing challenges. Usually, smaller iterations of a process are planned so that “tight feedback loops” can allow for quicker, more efficient adaptation.
Then the software engineers analyze what’s needed to create the elements that will meet the client’s goals. After that, they make their design, using whatever technologies and third-party tools they need. In other words, they build the product. “Then we are exposed to the harsh reality,” Miro says, “that we’re doing a very complex business, working on the code base. There’s a lot of complexity there. From time to time, we face issues, like maybe the third-party tool doesn’t work as described, or maybe it doesn’t work at all. And prototyping toward a solution without knowing all the variables is risky. While we could go back to the waterfall approach, where we try to research and design every part of the system before the implementation, it's often more efficient balancing the two.” The goal here is to avoid a significant delay in a project’s life cycle without hindering the original plan; therefore, compromises must be found between the third-party tools and new, custom tools to get things where they need to go.
The next phase is code review, which is exactly what it sounds like: Other engineers examine the coding work done so far to make sure it’s in good shape, to catch and correct mistakes, and to see if anything that should be addressed has been missed. Miro says, “It’s a great chance for other team members to gain knowledge about the functionality that’s being built, because in software, we build a lot of small components of the system, and it makes us much more efficient when we can actually reuse some of the existing components to build new functionality more quickly.” It also improves workflow efficiency for all team members.
After the code review, quality assurance engineers step in to test the product as if they’re the end user. This process can be manual or automated, allowing the engineers some flexibility in their approach when reviewing such elements as performance and security, among others.
Finally, after the code review is completed and approval given, the product is shipped (that is, put into production and sent to users). “Now we have to maintain it, because now it’s part of the system,” says Miro. “The needs for the product will change, and we need to keep this functionality up to date to suit our goals in the future as well.” This can present a challenge — these products aren’t static, and when they grow, the complexity of the entire system also grows. “As the system size grows,” Miro says, “as we add new functionality, velocity doesn’t only decrease. It decreases exponentially because each connection and dependency in the system is like adding another node to the network.”
What this means in practice is that the software engineers have to consider how their new functionalities will mesh with existing functionalities while they’re building. They also have to consider that, as their system grows, it will require more people to maintain it. “If we had a team of two engineers at the beginning, and now the project grew 10 times over a couple of years,” says Miro, “two engineers can hardly keep all the parts of the system in their heads throughout that time.” His teams deal with this challenge by using software architecture, he says, which “makes the boundaries between the small parts [of a project] really strong.” This approach has to be taken at the beginning of a project in order to work.
Engineering the Process: Goals and Metrics
For Miro and his teams, the definition of success is simple: “Our goal is to continuously deliver stable software fast.” Time and reliability are key factors in every job. “The approach can be summarized as saying that teams that ship more often ship faster and ship more-stable releases, because if we ship more often, we are shipping smaller parts of the system, so there’s less chance of systems breaking. Also we usually have multiple engineers working on the system. And if we ship more often, there’s less conflict between the work that different engineers are doing and the communication gets easier.”
Pointing to the work of Peter Drucker and Richard Hamming, Miro says, “You get what you measure. If you measure the wrong things, you’re going to get the wrong outcomes.” So the engineers take the time to think about the metrics, once they’re determined, in order to make the kinds of changes that will give results. If they want to ship frequently and minimize errors, they need to know what kinds of things they can do to reach those goals.
There can be different successful approaches to this problem. “We can try to tweak the process,” Miro explains. “We have that plan, build, ship process, and we can play around to see what brings the best outcome. For example, we can switch the code review and the quality assurance process. So if we spend a lot of time in code review and then the quality assurance engineer tells us that this is not really fitting the requirements, we have to go back, make changes, and now we’re back at code review. We’re spending double time there. So maybe it makes sense to flip the two and have quality assurance done first.”
Engineering the Organization: Conway's Law
Conway’s law states that any organization designing a system will ultimately create one that reflects its own communication structure. “When I first read this, it seemed very far-fetched,” says Miro, “but it has been proven over time.” He cites this as a reason that software architecture is so crucial. Large teams tend to create large systems, which can slow everything down and create a lot of dependencies. “Now we have the burden of a big system,” Miro says. “No matter how many workers we put into it, we’re only slowing it down. What we actually want to do is design the organization of the team so that we can break apart the software architecture into manageable parts in terms of complexity, and then each team might have a chance to be productive over a long period of time.”
The Skills to Succeed
Engineers have to be both technically minded and creative to navigate the many demands of their jobs. But for Miro, critical thinking is key. He believes that the best engineers know the theory behind why you’d choose one option over another and also know what negative outcomes to watch for. “When people say ‘engineering mindset,’” he adds, “I usually think that [they mean] analytical thinking and the approach to solving the problem.” This skill is honed over time, of course, as engineers are confronted with real-world problems in their jobs.
Whether you’re a software engineer or a quality assurance engineer, each step of the learning experience engineering process is complex. “These problems involve a lot of parameters. There is no recipe you can just follow. You have to adjust all the time,” Miro explains. “I’m happy to say I’m surrounded by engineers who can make good calls on these decisions and impact our outcomes positively.”