Upgrading Our Infrastructure
Posted on 2021-06-07
As many users may be aware, Simply Fortran is built for three different and distinct platforms and a variety of operating systems. Our product started off as a Windows-exclusive development environment with pre-packaged compiler that was designed to address the complexity of setting up a Fortran environment on this decidedly non-UNIX operating system. In late 2015, we first released our GNU/Linux version of Simply Fortran, running natively using the GTK+ toolkit. In late 2016, this GTK+ version of Simply Fortran also made its debut on Apple's macOS. We quickly went from building releases for a single platform to three basic platforms, though our GNU/Linux support required compiling for, at the time, five separate operating systems.
Having all these separate platforms required a solution that didn't involve manually sitting down at each supported operating system and manually launching builds. Of course, our problem isn't unique by any means, and we decided on using a "continuous integration" package called Buildbot that happily ran on all of our supported platforms. Every night, our build servers would start compiling Simply Fortran for Windows and GNU/Linux based on our current development branch of Simply Fortran's source code. When a release was deemed ready, we'd instruct the system to launch builds across all our build servers, including a Windows machine, an OS X machine, a Raspberry Pi, and no fewer than four GNU/Linux virtual machines. Our current collection of build servers have changed over the years, whittling down the GNU/Linux virtual machines to two while increasing our macOS servers to three, but the general concept and approximate number of builders remains the same.
Over the years, however, the Buildbot project advanced, leaving our configuration files and build instructions tied to an old release series, 0.8, which only ran on Python 2.7 or older. Almost every GNU/Linux distribution still includes Python 2.7, and it continues to be simple to access this version on Windows and even Intel-powered macOS machines. So we honestly deferred maintenance of our build infrastructure, relying on the continued availability of Python 2.7 and this old Buildbot release.
Two things complicated this strategy within the last few years. First, Python 2.7 support officially ended at the beginning of 2020. While it will quite likely continue to be available for years to come, there are concerns that Python package installation will eventually break entirely. Second, Apple introduced a new CPU, the M1, that brought macOS to a non-Intel architecture. Trying to build and install Python 2.7, which does not expect an ARM64 CPU under macOS, and Buildbot 0.8, which is hopelessly outdated, on these new Apple machines was an exercise in futility.
While we could have attempted to upgrade to a modern Buildbot version, much had changed in that project, leaving our build configuration file somewhat useless. Though we could try this upgrade, there would be no less work in switching to an entirely different continuous integration package. Additionally, our needs only represent a fraction of the functionality most continuous integration systems provide. Faced with setting up a complicated, unfamiliar client/server system, we opted for a more novel approach.
Simply Fortran is now built using a pure-Fortran continuous integration system called Levitating. Right now, the system is in its infancy, but it does indeed work. Version 3.19 of Simply Fortran was built for all platforms using this solution. The project maintainers just need to author simple JSON instructions for any desired build operation, called a "Job," to be assigned to "Players," or the client computers that will perform the build task. On those "Players," the instructions are followed step-by-step to produce and upload all the installers that appear on our Download page.
The Levitating project is still in early development, and many, many features are missing. For example, there is absolutely no security within the server interface; anyone can launch a Job or pretend to be a Player. As of today, it still doesn't support timed builds, like launching instructions at a certain time every day. The project does continue to expand as we add these features to, at the very least, make our jobs at Approximatrix easier.
At face value, Levitating does demonstrate applying Fortran to a computing task for which it is rarely used. The Fortran clients and server all wrap common C networking functions and call out directly from Fortran code to transfer files, report results, and interact with users. To provide this functionality, the project includes a rudimentary HTML template system which is, again, written entirely in Fortran. Though perhaps some of the chosen technologies are dated, such as using the common gateway interface for a web front-end, the system should be usable for many years to come.