Scheduling: Schedule, design, code?

Or perhaps: schedule, design, build?  Either way, while that sequence may sound like waterfall, even agile is really a repetition of this sequence over and over.  This page discusses why these 3 steps are problematic for software, but still must be followed despite problems, and how to schedule projects using this sequence that initially can appear to be broken.

Contents:

• Build vs Engineer: Which Describes Software Development
  • Background
    • Engineering Design vs Artistic Design
    • Engineering Design: A tautology?
  • Construction vs Engineering: What’s The difference?
• Software Development: Construction Model vs Engineering Model
  • Construction Model/Waterfall Advantages: Scheduling/Diversity
  • Engineering/Agile Advantages and Disadvantages
• Rethink: Scheduling Engineering vs Scheduling Construction
  • The problem
  • The proven solution
  • Applying the solution to software
  • Scheduling with Scrum
• Conclusion: How To Schedule, Design, Code

Build Vs Engineer: Which Best Describes Software Development?

To be able to schedule the completion of a task, it is useful to consider the nature of the task and the nature of the steps within a task.

Design: Engineering Design vs Artistic Design.

The verb design can be interpreted in subtly different ways.  The design of the Rubik’s Cube by Erno Rubik is an example of engineering design, not of artistic design. But someone can make a new Rubik’s cube with a new artistic design. When Engineering design is consider  novel the designer can apply for a patent, while with artistic design the designer considers their work protected by copyright.  The nature of the intellectual property created by different design types is sufficiently different that there are different ways to protect that intellectual property.

The engineering process as describe here gives some insight.  Note the iteration, with no way of being certain how many iterations are required to produce a result.  The implication is it is impossible to be certain how long the process will take.   However,  in most cases, with artistic design, a single iteration is sufficient, making the process easier to predict.  Depending on the task, software design can be a about artistic design, engineering design or mixture of both. In reality, the design tasks are almost always engineering design.

Generally software is written because software to do the task does not already exist. Unlike with a chair that you might build because you need another chair, software is generally no written because you need an exact copy of software that already exists.  This means there should always be engineering design required, which in turn suggests there could be a highly variable time to predict how long the process will take.

Engineering Design: A tautology?

Engineering is generally described by the dictionary as both design and building. The dictionary definition suggests a waterfall sequence, while the flow from an engineering perspective clearly conveys a more agile approach.  Regardless of the sequence, there is both design and build, however for engineering the substance is in the design.  Consider the phrases ‘engineer a bridge’ and ‘build a bridge’  as we use them today. To engineer a bridge is to either design the bridge or some detail of the bridge, while to build a bridge implies no design work is required.  As we use ‘engineer’ in this sense today, an engineering project is considered complete when the design is complete.   Building that is necessary to test the design or to be ready to test the design could be part of engineering. Once a design is complete, in todays world we then normally use ‘construction’ or simply ‘building’ and keep ‘engineering’ for when there is a new design.

Construction vs Engineering.

While both ‘construction projects’ and ‘engineering projects’ can involve building something and will need a design- the construction project is primarily all about the building, and the and the engineering project is all about the design.  A construction company will often outsource the design to an engineering company when custom design is required, and then commence construction with all design work complete.

Consider building Swedish designed flat pack furniture.   All that is required by the consumer is building or construction.  Engineering is what took place at head office to produce and test the original design. But even within that original engineering project there a simple construction components, because the design of that component already exists, and is already tested.

Construction is where the overall design already exists and is already considered tested. Engineering is where the overall design is not yet tested, although there will normally be components of the overall design that are tested.

Software Development: Construction or Engineering?

The waterfall development process works on the principle that the software development can work like construction, and all design is already complete and effectively tested.

In contrast, agile is based on considering software as an engineering project, and that even components within that engineering project are themselves engineering projects.

Waterfall Advantage: Scheduling

As in a construction industry project, there is still the initial engineering/design phase which can be very difficult to schedule. With the construction industry the main costs and the major time for the project all take place one the design is agreed, so the variable timing of the shorter, lower cost engineering/design phase is not that significant to the overall project.

Construction itself is following known steps, so it can (at least in theory) be accurately scheduled and costed.  In theory, applying this model to software could provide not only accurate overall scheduling, but with a set of discrete steps, accurate tracking of project progress.

Waterfall Advantages: Skill Diversity and Low Cost

Again consider the construction industry.  The design phase requires architects and engineers who are highly educated and costly, however most of the actual work can be done by labourers who are less expensive labour.  The construction phase still needs project managers and foremen, but few in proportion to the labourers.

Applying this same model to software results in software architects and business analysts for the design phase, and while the actual development will require project managers and team leaders, these will be few in relation to more junior programmers, and these programmers can possibly be outsourced or even off shored to lower costs.

The advantages and disadvantages of Agile

Agile has one simply advantage:  the engineering metaphor it implements actually does apply to software.  This gives the advantage of actually confronting reality, while the waterfall approach has attractions,  as a model for software it is fundamentally flawed since design is required during the ‘construction’ phase.  The result is any supposed advantages are not realistic.

The disadvantage of facing reality is that 1) it become clear that it is impossible to guarantee a schedule for a design that is not yet complete, and the design will not be complete until the software is complete, and 2) the skill diversity approach is broken as with design needed at all stages the low cost ‘labourers’ will still need to make design decisions and the impact of these decisions is significant.  So no exact scheduling of construction, and the separation of roles to analysis, programmers, project managers is flawed and at the very least needs each skill in every team, if not every team member.

Summary: There are question marks around whether software can really be reduced to the waterfall model, and that would be the only way to enable reliable time estimates.  The implication is that in project discoveries must impact schedules and cannot be avoided.  Something has to give.

Rethink: Scheduling engineering vs scheduling construction.

The Problem.

The mindset of construction is that at the project outset it becomes known what is to be done, and then good planning will result in an accurate timeline for the project.

However software cannot generally be properly reduced to the construction model,  leaving an engineering approach where it is never known exactly what is to be done until the goal is achieved..at which point the project is basically complete.

The Proven Solution.

Consider car manufacturers.  They construct cars. In advance of manufacture they know very accurately what the specification of each car to be built will be, and how long to build each car.

However, now consider the engineering aspect of a car manufacturer: designing new cars to then be manufactured.  Design, prototyping and tooling.  As an engineering task, on the basis of theory described here, the time to design a new car to a given set of specifications can be estimated (educated guess), but there is insufficient information for an accurate figure.  The solution: something has to give.  Either the exact specification, or the exact amount of time must be adjustable.  Given many manufactures desire a new model each year, the time is not adjustable, so the result is the specification becomes flexible.

A fixed amount of resources is allocated to engineer improvements to the current model.  All improvement ready in time become part of the new model,  There is a list of desired improvements and ideas for improvements and these can be prioritised. The limitation is that the exact set of features that will be ready for the next model is not known at the start of the fixed length project. Only those that can be completed in time will be in the next model. Usually there will be more features and improvements identified as desirable than can be ready in time for that next model.  A list of features and improvements which are thought can be ready for the next model is made, and work starts on the list.  If a feature/improvement is not ready in time, it will have to wait as the release date for the next model cannot be pushed back.he projects include is

Applying the solution to software.

Agile allows taking the industrial engineering approach, and applying it to software.  Projects like Ubuntu Linux and now even Windows 10 releases industry have new software releases at fixed intervals.  The product versions even are based on dates and those dates are declared at the project outset.  There are have been two Ubuntu versions every year since October 2004 (4.10 Warty Warthog). One version in April, with the version number of the year and then ‘.04’ (April is the 4th month so 5.04 was in April of 2005) and then another in October with ‘.10’ for the tenth month. How do they keep such reliable schedules? Features that do not make the deadline are pushed back to the next version, that’s how.

Scheduling and Scrum.

The scrum process is like a series of mini-releases, with the completion of each sprint resulting in a new set of stories, tasks and bug fixes ready working tested and integrated into the system. Scrum planning can take the same type of approach with issues not able to be complete in time pushed back to a future scrum.

The danger is that if not managed correctly, an individual issue could absorb the team for entire time allocated to the sprint.  If that issue still cannot be completed, then there is a sprint with nothing completed.  The solution is to budget time for each issues.  When that budgeted time has elapsed, the issue should be reviewed.

The choices for the review are

1. divide this issue into parts and push what cannot be done in this sprint back to the backlog (which could even be the entire issue)
2. push other issues to the backlog to free time in this sprint for another allocation of time to this issue
3. both of the above.  Push part of the issue to the backlog, but still allow a new block of time for this now simplified issue and push other issues to the backlog to free up time for the now reduced issue

Conclusion: Schedule, design, code.

Sprints should be set with fixed end dates, or at least end dates that will have only a small window of variation.  As the window approaches, new tasks are pushed back in place of being started when there is insufficient time and only tasks near completion can affect the sprint close date within a predetermined window.

Each task can be scheduled before the task is started, and the schedule should allow a high probability the task will be complete, but this cannot be guaranteed. At the end of the schedule time the task should be reviewed.  Either this task is pushed back, another task is pushed back, or the sprint will be late.

So it turns out that the sequence: Schedule, Design, Code, can work.