Project management is a crucial weapon in the arsenal of any software development outfit. Its probably the most-discussed competency in software engineering, judging by the sheer volume of scholarly papers, conceptual models, blog articles and entire schools of thought that have been churned out on this subject over the past two decades. We’ve seen process frameworks like Waterfall, SSADM and RUP come and go, and a shift from centralized delivery responsibility resting on the service provider, towards distributed ownership across an extended team inclusive of the client. We live in a world of “Agile” software development today, a zeitgeist of management thinking patterns that are based on keeping processes to the bare essential, building products incrementally and eliminating humbug within teams. We have even seen the formal “role” of the project manager (stereotyped as the big, bad bogyman in the team) disappear within the modern agile paradigm.

Call it what you like, a person or the collective reasoning within a team, we find that effective project management remains an essential ingredient to “getting the job done” . Moreover, project management success in software development engagements often remains illusive. I’d like to summarize our own successful methodology at Calcey, and go on to explain a few of the deeper lessons we learned through our collective management experience, for the benefit of our future clients.

We follow a project management methodology that is a derivative of Scrum, which has benefited from long years of practical experience in delivering projects of varying sizes and technical complexities. Our conceptual framework is fairly simple. We agree with our clients to form a single team having joint responsibility for the project, at the early stage of pre-sales negotiation. Whilst in theory we are not supposed to estimate the end-to-end scope of work in Scrum, in practice we have found it impossible to find a client who would agree to an entirely open budget and no indicative calendar timeline for building a product. So an initial ballpark estimate is made. This is purely for purposes of budgeting, to provided the client with a broad feel for the costs involved and to determine the resource bandwidth to be deployed in order to meet a very approximate calendar schedule. This sort of budget is made against the broad set of features that the product comprises of, as understood at the inception of the project. Once a project is contracted, we move forward in earnest to apply our Scrum model.

A Calcey Scrum Master’s life revolves around their project backlog. They manage both the product’s roadmap of features and the specific tasks (or bugs) for the current sprint via an enterprise backlog app such as JIRA, TeamworkPM or Basecamp. JIRA offers the highest flexibility in managing the complete life-cycle of a development project, but both TeamworkPM and Basecamp have proved to be interesting alternatives to managing smaller-scale engagements. In any case, it is not the choice of the tool itself that we found important, rather the diligent use of the backlog as a concept for task management that helped us most. Handwritten backlogs diligently maintained in the corner of a whiteboard tagged with the words “don’t erase” seemed to work better in some situations!

We plan development for a time-boxed sprint, whose duration is usually a fortnight for technologies that we are well experienced in, and a month for greenfield technologies or projects of high engineering complexity. The duration would be decided at the initial sprint, where we estimate what could be achieved in the first sprint, within the budgeted engineering bandwidth. Once decided, we stick to this time-box throughout the lifetime of the project. The outcome of any given sprint is of course a release of working software –  working but not bug free or complete in functionality. As the sprints progress, the software “emerges” as a viable product for launch. A lot has been said in the industry about the generic form of the Scrum methodology, so I’d like to move on to a few specific lessons we learned at Calcey through our experience. A snapshot of the recurring activities that we practiced shown below.

The first and biggest lesson learned for those of us who were new to Scrum was that, unlike any other methodology, Scrum is an explicit activity like coding or testing. We’d scan the client horizon as well as our own engineering backyard each morning via the daily stand-up meeting, update our project backlog, and get into action to follow up on the individual tasks that need facilitation. We found that if we have a “living”  task backlog that gets updated without fail each day (with dates, milestones etc), we then could use it as the vehicle to drive our work; to psyche up the team, provide expert external assistance or reset client expectations. So the Scrum Masters don’t “go to sleep”  when not at sprint planning and the daily stand-up meeting –  on the contrary, they work hard each day to facilitate the resolution of issues arising from the daily stand-up meeting.
The effort required for effective sprint planning is not trivial, as we learned through experience. In theory, the estimate given at sprint planning (“I’ll finish task X within the next two weeks”) is considered sacrosanct. This aught to be, because cascading task “spillovers” into subsequent sprints could buckle the whole paradigm of time-boxed incremental achievement, and sprint velocities could take a nosedive. So what we found was that it was worthwhile to invest an entire day for sprint planning. This day is not counted into any given sprint, and the day’s principle goal is to freeze a list of tasks to be completed during the upcoming sprint. A full day provides enough time for the team to mull over the complexities of the tasks and break them down into smaller goals if necessary. The sprint planning meeting itself assumes a sort of “workshop” format where folks could pop outside for a quick R&D and return with better knowledge about the complexities of the work involved. Ultimately, everyone walks away to implement what they would consider as their own sprint plan, approved by the product owner.
Another significant learning was overcoming the common problem of trivialization of testing. The updating of automated unit tests and manual test plans, smoke testing, regression testing and bug fixing all take up a considerable percentage of the time taken for implementing a given functionality. Moreover, contrary to the idealistic belief amongst agile gurus that all competent software engineers are also competent testers (or aught to be such), we find in practice that the eyes of a person with a strong end-user perspective is essential to ensuring a healthy demo at the end of the sprint. So we found it useful to divide a given sprint timeline conceptually into a “new dev”  period and a “testing and bug fixing”  period, at sprint planning itself. This helped us to reduce the otherwise frightening tendency of “bug pileup”  that so often happens in Scrum projects –  where new development forges ahead of bug fixing, causing instability in the releases as time goes by.
The use of lifecycle automation tools was an immense help to us, and we considered them as part and parcel of our agile methodology. Anything useful ranging from build automation tools like Cruse Control, source repositories like GIT, test scripting frameworks like Selenium and Code Analyzers like FXCop were absorbed into our development framework.
The single hardest challenge though was making sure that the client representatives became an integral part of the team, and that they felt inherently responsible
for the incremental development in a hands-on fashion. Success or failure of a given sprint would be declared by the product owner immediately after the sprint demo; this is one important reason why the product owner (or his or her competent representative) is part of the team. This helped us prevent a situation where a client determines the product under development has veered radically off course after (say) a dozen sprints. Because if such a situation does arise, it basically tells us that the fundamental paradigm of dealing with a complex problem in small increments has not been adopted by the client. There are many ways to convey the message of joint ownership and incremental assessment, and in practice we have found that the most effective way is to discuss this very problem upfront prior to undertaking a new project. We usually stress the fact that the success or failure of each sprint must be determined at the next sprint planning, and adjustments must be made locally, at the scope of each sprint. These adjustments include making “management decisions”  like filling skill gaps or increasing the engineering bandwidth.
Lets face it, software development is not comparable to dam building –  a common misconception amongst management types. Although there is definite commonality in the broader values that are required of the team like honesty, dedication and professional competency, the fundamental drivers of work are not exactly the same. Intellectual effort taxes both our left and right brains equally with plenty of logical reasoning bootstrapped by flights of inspiration and lateral thinking. This “mind game” of software project management requires a management methodology that fosters creativity, whilst compensating for common human failings like poor memory, to drive it forward. Scrum is just such a methodology, and has proved highly effective for us at Calcey.

What makes an applicant for a software engineering job an attractive candidate for hire? This is a perennial question that is likely to be uppermost in the minds of both employers and their prospective employees. Software engineering is highly competitive job market where, on the one hand, industry expectations run high, and on the other hand we have hundreds of “education providers” professing to deliver a sound education in computer science. It can get confusing to aspiring software development professionals as to what exactly is required of them to secure that first salaried position… and it can be frustrating to employers when newly hired staff are not performing up to modest expectations simply because they have not approached the trade with the right mindset.

Let us try to outline the measurable skills of an “ideal” software engineering candidate who presents herself for interview. First and foremost, she’d be very strong in her software engineering concepts. An in-depth knowledge of OOP concepts would be evident. She’d be thorough with the architecture of at least one development platform of choice, and be aware of how the various sub-systems and libraries of the platform work in cohesion. For example if we consider .NET as the development framework, a savvy software engineer would be aware of the high-level workings of the CLR and common language infrastructure. The candidate would have the essential background technical knowledge to develop Web applications such as an understanding of the lifecycle of a Web Request. She’d be familiar with Lifecycle Management concepts like Continuous Integration, Unit Test Driven Development and Configuration Management.

A “good” software engineering candidate would know about industry-recognized Design Patterns, and ideally have adopted a few patterns into their experimental project code. Conceptual knowledge of widely used Design Patterns like Singleton, Abstract Factory, Factory Method, Facade, Proxy etc is essential knowhow, since these concepts tune the programmer’s mindset to leveraging best of breed solutions to standard design problems, without reinventing the wheel (and reinventing it rather poorly perhaps). Whilst generally having a modern, agile, paperless, code-design based approach to programming, a budding developer must have enough sense to initially whiteboard any non-trivial implementation, and seek peer review from her teammates. At an interview, this skill could be displayed through one’s ability to represent a simple design problem as a UML class diagram.

In this day and age of vastly scalable non-relational databases and cloud computing, one might be tempted to frown upon knowledge of plain old-fashioned relational database concepts like ER diagramming, SQL, Normalization and Optimization. However, these concepts are still very much in use in most enterprise applications, and must be studied and understood. Woe to the interview candidate who cannot answer a question like “Why would one sometimes need to de-normalize a database? Can you think of an example?” or “What is the use of an Index?” .

Some conceptual knowledge of how people work together to deliver a project to a client is mandatory for beginners. Familiarity with a lightweight team-engagement paradigm like Scrum would be ideal to have.

Successfully facing a test for conceptual knowledge is of course not the only indicator of potential success at an interview. But the concept-savvy candidate is the quintessential software engineer. The era of concept-blind “code monkeys”  working principally through trial and error (“copy-paste coding”) is to be frowned upon by any respectable software engineer or employer. Of course, we are speaking of engineering graduates who have had ample time and guidance to perfect their attitude to programming. The story might be different if one is a job applicant fresh out of high-school. In a sense, the evaluation that such trainees would have to face would be harsh, because employers would have to simply rely on the rather controversial concept of “a high IQ” . It has to be admitted that a good developer will almost certainly have strong innate logical reasoning capacity, usually expressed as a gift for providing algorithmic solutions to real-world problems. This is the sort of test that one who has had no formal training in computer science would face at an interview (e.g. “tell us the steps to sort a randomized list of numbers from 1 to 100” etc).

Having said this, possessing a high IQ, being knowledgeable about engineering concepts and being a great team-worker still does’t complete the picture of a “good” software engineer. At the heart of a successful developer lies an appreciation of the end user’s objectives for the system under development. No matter how “smart” a software developer is at codifying algorithms or grasping new frameworks, she only succeeds when she has met the end user’s expectations. So “well rounded” developers are also great end users –   ” those who tryout new apps as hobby. Unless you use software freely in your own life, its unlikely you know what is a user-friendly, defect-free app. Along with an appreciation for usability comes an appreciation for testing one’s code.

So from the perspective of a job applicant, what does one do as prep for that first interview? Ask oneself the question, am I aware of the basic concepts? OOP-check. Design Patterns-check. Web Request Lifecycle-check. Logical reasoning and writing of algorithms-check. Configuration Management concepts-check. Relational Database design-check. And so the list goes on. Freshers preparing for an interview should master the ability to clearly demonstrate one’s knowledge of these concepts, by giving both textbook metaphors of their use and purpose, as well as recounting how one has used or encountered these concepts in past projects (at University or elsewhere, perhaps within common development frameworks itself).

Let us now look at the other side of the coin –  how does a potential employer spot someone who has great potential as a software developer? At the heart of making a good hire lies the ability to gauge if a candidate can reason from first principle, literally thinking on her feet. After making a candidate comfortable, its an excellent idea to present a simple real-world problem on the board, and see how far she can get with providing a solution in design / pseudo-code. Give the candidate ample time, support and encouragement. Once the initial jitters are at bay, a good candidate will always make a genuine attempt to answer the problem. A weak candidate will always stall early with algorithms or design problems, because of a lack of critical reasoning capacity –  an essential trait for becoming a good software engineer.

It might sound like a harsh reality, but software engineering is not for everyone, just like any other “trade” like art or music or management is not for everyone. However everyone must be given the opportunity to try their luck at this fascinating trade, and it is likely that those who succeed will be those who approach solving real-world problems logically from first principle, with great diligence. The sooner aspiring software engineers learn this, the better their career prospects. This is, and must be, a transparent truth between the industry and the thousands of prospective new entrants.