… because if you don't you are dead. Dead in the gutter dead.
-- Charles Bronson in Kinjite
Emergent design is a buzzword among both developers and managers. Design and code is supposed to emerge through trial and error by refactoring. If you don't accept it unconditionally you will most likely be seen as a Luddite from the Stone Age.
I can't really say that the idea is wrong since it often works for me when solving design and programming problems in an area where I don't have a whole lot of knowledge. However, what is wrong is that if you even dare to question the idea of emergent design in an environment populated by emergent design drones you are dead meat.
I believe many IT shops have never really tried to understand what emergent design really should mean, when it should be used and when it should not be used. Unfortunately many design philosophies such as emergent design, test driven development and others are viewed from an almost religious perspective. If you don't submit to the doctrine of the place and time you'll become an outcast because just as with many religions, trying to understand and analyze what's going on is blasphemy.
Of course, emergence is just a fancy word for something that I could not predict. But hey, maybe someone else could have predicted it! The fact that I might not be able to predict something but someone smarter than me can predict it is the key to understanding when emergent design is a good thing and when it is not.
Because emergent design is directly related to someone's ability to predict, it's pretty obvious that emergence is not an intrinsic property of some phenomena. Instead, bluntly said, emergence is simply ignorance of someone on which some phenomena crept up on!
Now when knowing that emergence, just like many other concepts and phenomenon, is just another relative notion I should be able to put that knowledge to good use. Because being able to predict something is usually preferable to having something emerge upon me, one idea would be to find methods which help me predict things. In the IT world this is a little tricky since most useful systems have lots of moving parts making it difficult to know in advance how to piece them together. What makes it even worse is that the parts are not all the same, most parts have very different properties making it even more difficult. Unfortunately there are no general methods for designing things that have lots of parts and where the parts are very different from each other.
Fortunately, there exists a simple solution to the conundrum of being able to predict the future, it's called called domain knowledge. Domain knowledge is almost unbeatable when it comes to creating complex designs that often come close to what I want. In my mind, domain knowledge is almost magical since I can often leap frog far ahead of what I could ever dream up myself.
The question now is why domain knowledge is so powerful? It is actually pretty simple. Men with long gray beards have been sitting locked up in cubical for decades thinking up solutions to problems over the past few thousand years. Women (without beards) of course have done the same. Myself, an IT guy, I would not only be dreaming if I thought I could beat a domain expert on his or hers home turf, I would simply be a fool. Unfortunately, it's more common to experiment with designs and code in well-researched domains than it is to spend a day or two reading up on the basics of the domain before pounding away on the keyboard.
Now, let's assume for a second that I actually could come up with solutions that were on par with the solutions from experts in some domain. Unfortunatly, most likely I would still loose out over time. The reason for this is that domain experts have knowledge that not only solves problems in a domain, but they also solve them in such a way that the solutions will easily absorb new requirements and problems showing up later on. The concepts and procedures within a domain don't only solve a single problem, but are designed to solve a host of problems in a consistent way within a domain.
Because domain knowledge has been created by lots of people through both hard thinking and trial and error methods, domain knowldege effectivly is equivalent to massive refactoring of ideas. So, when I use domain knowledge I am simply skipping the refactoring procedure, or at least part of it.
Because domain knowledge has been created by lots of people through both hard thinking and trial and error methods, domain knowldege effectivly is equivalent to massive refactoring of ideas. So, when I use domain knowledge I am simply skipping the refactoring procedure, or at least part of it.
A simple example that shows how far domain knowledge can get me with a minimum strain on my neurons is in the domain of algorithm design. Here I'll take a quick look at the design of algorithms where I find solutions by enumerating potential solutions systematically one by one. The idea is of course backtracking where solutions are found through a depth first search in a solutions space generated on the fly:
void search(partialSolution,nelements,auxData){
if(isSolution(partialSolution,nelements,auxData){
processSolution(partialSolution,nelements,auxData);
return;
}
potentialElements=createNextElements(partialSolution,nelements,auxdata);
for(el in potentialElements){
addElement(el,partialSolution,nelements+1,auxData);
search(partialSolution,nelements+1,auxData);
removeElement(partialSolution,nelements+1,auxData);
}
}
For example, the code to generate all permutation of an ASCII string with the condition that a character must be followed by a character of the opposite case becomes trivial. I assume that the string only consists of the characters [a-zA-Z]:
// mark functions
void mark(char&c){c|=0x80;}
void unmark(char&c){c&=~0x80;}
bool marked(char c){return static_cast<bool>(c&0x80);}
// str is the input string, res is a temporary string used by the algorithm
void printStrPerm(ostream&os,string&str,string&res){
// (1) check for solution
if(str.length()==res.length()){
os<<res<<endl;
return;
}
int resind=res.length()-1;
// (2) check if there are no valid permutations
int nu{0},nl{0};
if(resind<0){ // first character in result?
// we only get here once
for(int i=0;i<str.length();++i){
if(isupper(str[i]))++nu;
else ++nl;
}
if(abs(nl-nu)>1)return; // return if there are no permutations
}
// (3) loop over remaining characters
for(int i=0;i<str.length();++i){
if(marked(str[i]))continue;
if(resind<0){
// (4) check if we can prune directly on first character
if(nu>nl&&islower(str[i]))continue;
if(nl>nu&&isupper(str[i]))continue;
}
else{
// (5) check correct case of next character
if(isupper(res[resind])){
if(isupper(str[i]))continue;
}
else{
if(islower(str[i]))continue;
}
}
// (6) run back tracking algorithm
res.push_back(str[i]);
mark(str[i]);
printStrPerm(os,str,res);
res.pop_back();
unmark(str[i]);
}
}
Without pruning the algorithm would have become just a few lines at the cost of execution speed. I could have removed all the pruning and just checked at (1) if I had a valid solution. There are also a few programmatic optimizations that I could have done if I really wanted to squeeze the execution time.
Because I've pruned all invalid permutations, the only thing to check at (1) is if the result has the right number of characters. The code at (2) only happens ones and checks if there are any valid permutations at all. If not, there is no point continuing. At (3) I loop over all characters in the input string and filter out already used ones inside the loop. At (4) I prune if possible on the first character that could be put in the result string. That is, if there are more upper case characters, I must start with an upper case character. The code at (5) checks if the next characters has the right case. Finally at (6) I have a character that is appended to the result string and sent off into the backtracking algorithm.
The number of permutations of a string having u upper case characters and l lower case characters are:
- 2 * u! when u == l
- u! * l! when u == l+1 (and of course the opposite when l == u+1
- 0 when abs(u – l) > 1
Yeah, I know the problem is simple and maybe there is a smarter way of solving it. But, the point here is that I can solve a problem and having confidence that the solution is probably a good one since I'm reusing ideas from a domain in which lots of people have done something similar. More importantly, I can be lazy and I don't have to think very hard.
Of course emergent design does not end with domain knowledge. Gaps such as technology mappings will always be there and has to be filled in. The trick is to know when to stop using domain knowledge, when to stop thinking really hard and when letting the design and code emerge.
