Binary files and revision control

I manage the SVN revision control server for the company I work at, which is mainly used for software and HEX files of firmware.  After a few mess-ups and wrong files released to manufacturing, the electrical and mechanical engineers saw the benefits of SVN’s revision control and wanted to join the SVN party.  The problem with revision control of files generated by most design tools is binary files (Open a Word or Excel file in a text editor and you will see what I mean).  Software is mostly text based and lends itself easier to revision control since a simple text diff will show changes.  Most binary files I know of don’t have a diff tool, so knowing what was changed is a little more difficult.

Any one who has managed a project with multiple developers without revision control knows that copying and renaming folders, creating compressed folders for archives, working on a common networked directory or the multitude of other methods can be a nightmare.  With basic file management there is no revision history on what was changed, who changed it, what the old version between ‘releases’ looked like and tons of other issues.  Using version control solves many of these problems, however the issue of having a single binary file that can’t be merged with a modified version still exists and version control tools like SVN provide more tools to make solving this faster and with less headaches.

Now I’m not going to get into the huge benefits of SVN, there are lots of pages on that, just search “why use version control”, if you are here then you are thinking about version control. One method I developed to make sure that binary files don’t get stomped on includes one over-all rule and a few developer rules:

Overall rule: The version that is checked-in to the SVN server is the master version.

The SVN server holds all the information on files including edit history and commit messages.  If a developer wants to make changes to a file, update the local file to the version that is on the SVN server and then make changes.  Other developers may have updated the file on the server since their last update, the only way to make sure they have the latest version is to update.

Developer rules

1. Communicate with EVERYONE who will be using the binary files
2. Name a manager of high importance files
3. Be verbose in commit comments
4. When you are done with changes, commit them

Communication is key in working with binary files under SVN.  If someone needs to work on the binary files, talk to others who edit the files and see if they have updates to the files before starting work. I have seen a number of developers who think that version control solves all the problems, they just edit anything they please and commit the changes without communicating to others.  This may work for text based source files, but it will lead to problems with binaries. I have also seen engineers who talk with others on the team to make sure they don’t stomp an on going edit. “I want to update project XXXXX to have the new YYYYY, are you using it?” “Well I have some updates, let me check them in and I will email you, then it’s all yours”.

Naming a manager of high importance files (such as libraries) allows developers to check with a single person on updates and changes.  Being a manger of files comes with responsibilities. It is the managers job to make sure that changes to the files do not overwrite or conflict with others, and that the changes meet any requirements of the company.  Also, if there is a conflict with files the manager should be knowledgeable enough with SVN and the company process to resolve the conflict using the development tools.  This understanding should include making multiple SVN checkouts of different revisions and depth of knowledge in the development tools to bring multiple changes into a common file.

All developers should add verbose comments to the commit logs of SVN.  This seems to be a really difficult thing for lots of people, too many times I have seen “Changed stuff” as a commit message when in reality a commit message like “Added footprint for part XXXX to the library.  Updated text on YYYY schematic” would do.  Commit messages inform others (and yourself when you need to come back to a project after some time) as to what was done with the files. This is even more important with binaries than with text sources since diff tools might not exist to see differences between versions.

The final rule is to commit when you are finished making changes to a binary.  If developers totally overhaul a design and then sit on it for days and days, there is a good chance that someone else might make a change, which can result in heated ‘discussions’ on which version is correct and who should re-do their work with the new file.  If rule #1, good communication, is followed and commits are done in a timely manner many of the issues with binary revision control can be avoided.

Now, this isn’t a hard and fast one-method-to-rule-them-all way of dealing with binary files, however with smaller development groups this method seems to work well.  As with other ways of managing project files it can be difficult for developers to make sure they don’t stomp on other’s work, however SVN provides the tools to fix these issues in a timely manner.

I use SVN because it is open source and a mature product, however it isn’t the only one, just the one I am most familiar with.  Other similar revision control systems exist such as Git, Perforce, CVS, Visual Source Safe and others.  Choose one that meets your needs and can handle the files types you are working with.

1 Comment

Other Links to this Post

1. Binary files and revision control | myBitBox — November 20, 2012 @ 12:23 pm

RSS feed for comments on this post. TrackBack URI

Leave a comment