Introduction
When you design a software system, a common best practice is that you should not mix a functional specification with a technical design. That is how we started out designing data migration systems over 10 years ago. Our current best practice is very different: the best way to design a data migration system is to combine both designs in a single document.
At first, this may seem counter intuitive. So, why do we believe that designing data migration software is different and combining these designs in a single document makes perfect sense?
Functional and technical design
Let’s explore the difference between a functional and a technical design. In short, the functional design contains information about what the proposed software should do, while the technical design specifies how the software should be built. According to most blogs and articles on the subject, the functional and the technical design should not be combined in one document (see for instance https://www.techrepublic.com/article/functional-vs-design-in-documentation/).
The most common reasoning can be found, for example, at www.sinax.be:
“The scope of the functional analysis stops at how the software has to be written. Therefore, a purely functional analysis contains no technical details about the implementation of the software. Although these requirements are important as well, they must be included in the technical analysis and have nothing to do with the functional aspects.
The reason for this is actually very simple: the functional analysis is a reference document for all stakeholders. Here the functionality of the software prevails and not the technical implementation. The end users of the software cannot give feedback on the technical implementation (…) while they have to be involved in the functional aspect.”
We agree that a functional and a technical design are two different things, but there are a number of reasons why we believe that combining them in a single document actually improves the data migration process:
- There is less risk of deviations between the two;
- Functional understanding helps the software developers;
- The intended audience of the document is different when it comes to data migrations;
- Less redundancy equals more efficiency.
Less risk of deviations
A design is hardly ever final before the software development it describes is finished. Review, complications that come up during software construction (for a description see wikipedia.org/wiki/Software_construction) and testing of the product will inevitably lead to changes.
Comments from the customer are usually related to the functional design (we will come back to this below…), while software developers will come across problems in the technical design – this can easily lead to a situation where only one of the design documents is changed. And for data migrations it is especially important to keep them in line, because they match each other very closely, as can be seen in this example:
It takes a very disciplined designer to ensure that both versions stay in line! Why not make life easier by putting functional and technical design side by side? This minimizes the risk of deviations between the two.
The presence of the other design serves as a friendly reminder that both designs must be changed. No more “I must not forget to bring the other document in line.”
Functional understanding helps the developers
The most important reason why the common standard is to have separate functional and technical designs, is that the intended audience for the two designs is very different. The functional design is intended for the customer, who should not be bothered with technical details. Conversely, the software developers only need technical and no functional information. That is the theory, anyway.
We believe that this reasoning does not apply to software designs for data migrations.
Build a piece of software, run it without error notifications – the developer has done his job, right? Technically, yes, but what if developers have the functional description of what they are building at their disposal? That would help them judge their work even better.
Also, this helps designers as well: developer’s test results are more specific and comprehensive.
Data migration designs have different types of reviewers
Data migrations differ from other software developments in one important aspect: we are not designing software from scratch, we are describing the way the source database must be used to fill the target database. For this the main input comes from people with functional and technical knowledge of source- and target system.
This implies that the review cannot be limited to just the functional design. The technical design must be reviewed by the workshop members as well, because they need to see that the ‘what’ and the ‘how’ match and that the right tables and attributes are used. Moreover, during a data migration project most of the reviewers will have both functional and technical knowledge. So, for them the combination of this information in one document makes their task much easier.
Less redundancy equals more efficiency
When designing a data migration system, a lot of redundant information is specified in technical and functional designs. For example, the functional design describes the semantics of a specific target field and a mapping function containing source data. The technical design contains exactly the same information. This obvious redundancy is specific for data migration systems.
When combining the design documents, this redundancy is removed, making the entire design much more compact and less time consuming (thus, less costly!) to create and maintain.
Conclusion
General wisdom dictates that functional and technical design should come in different documents. When it comes to data migrations, combining the designs has significant advantages. Now it's up to you to discover if these arguments are applicable in your situation as well.