C No Evil - In Practice

John Regehr, an Associate Professor at University of Utah, posed an interesting question recently:

Your mortal enemy would like to ship a really great product in a week or two. Towards the goal of maximally delaying this product, you may inject a single C preprocessor definition into one of their header files. What is it?

There was also an interesting discussion on HN where a number of people offered their own suggestions which ranged from cruel to downright devious. I was curious to see what would happen if you were to inject any of these into the sources of some commonly used Open Source software. Which changes would cause a catastrophic failure to run, and which cause slightly more subtle runtime errors that permit reasonably normal use (albeit with the odd confusing moment).

First I selected a couple of programs I use fairly frequently:

Application Testing
GNU grep 2.9 make check after rebuild
Python 3.2.1 make test after rebuild
Emacs 23.3 mess around a bit (really scientific, eh)

Next I selected a number of the macros to test, I picked the ones which I thought were likely to build and run fine for the most part but introduce weird isolated errors:

 

#define unsigned signed
#define long short
#define volatile
#define continue break
#define struct union
#define if(x) if(rand()<0.0001 && (x))

Python

The location I chose to change was in Include/Python.h, right at the top. Either this was a bad choice or Python itself was a bad choice, because only one of the builds completed, and even then there was no effect to the results of the testing. Take a look at the screenshots for the failures, the failures are mostly at the same step in the build.

Macro Result
#define unsigned signed Build failure.
#define long short Build failure.
#define volatile Build succeeds. Tests fine too.
#define struct union Build failure.
#define continue break Build failure.
#define if(x) if(rand()<0.0001 && (x)) Build failure.

Oh well, onwards and upwards.

Grep

Again I chose one of the "included everywhere" headers - src/grep.h - and put my macros right at the top. The unadulerated grep actually fails `make check` because one test unexpected passes (word-delim-multibyte). I'll do as the message tells me to and report the failure, and for the purposes of this post I'll ignore the word-delim-multibyte failure.

Macro Result
#define unsigned signed Build succeeds. Tests fine
#define long short Build failure.
#define volatile Build succeeds. Tests fine
#define struct union Build failure.
#define continue break Build succeeds. One test (fmbtest) fails!
#define if(x) if(rand()<0.0001 && (x)) Build succeeds, all sorts of random runtime failures, brilliant!

Emacs

The file I changed this time was src/s/darwin.h. Unfortunately there's no automated testing for emacs, so when it built fine I just had to play around for a while. As I said earlier, not very scientific really.

Macro Result
#define unsigned signed Build failure (executable built fine, emacs is invoked to build lisp libs and dies)
#define long short Build failure
#define volatile Build succeeds, runs OK
#define struct union Build failure
#define continue break Build failure (executable built fine, emacs is invoked to build lisp libs and dies)
#define if(x) if(rand()<0.0001 && (x)) Build failure

Conclusion

Well this wasn't the greatest experiment ever, I was hoping for odd random behaviour and dramatic failures. Alas I was stuck with build failure after build failure. Perhaps the problem was that I wasn't devious enough with where the macros were inserted - the locations I chose tended to affect the entire codebase. I could also have selected the programs I was testing a little better, maybe a few more with automated 'make test' steps.