There were (for us) many firsts with the release of Starry Night Pro Plus 7. Exoplanets, data streaming, in-application updates, automatic crash reports - to name just a few. The scope of these new facilities and modernization of the old ones required tons of development time on the app itself. In addition, a lot of development had to be done on our servers too, a topic of a future article.

There is, however, one other area which has required many hours of development and use of a large number of technologies, most of them necessary to automate the process. This component is now integral to the lifecycle of SN7 and any future versions of our desktop applications. It assures that problems are caught early and that updates are tested before everyone gets them. I am talking about our Starry Night Beta program.

Mentioning beta testing evokes notions of release candidates, fixed release schedules and long delays between changes in the beta code and their eventual release to the public. We, of course, do our best to adhere to the wisdom of desktop beta testing. However, we also wanted Starry Night Beta to be more agile, and, above all, always alive.

Among other things, with the new automatic crash reporting and streamlined workflow of dealing with bug reports and user submitted support tickets, we wanted SN7 development to be as responsive to our users as we can make it. We scrutinize every crash report and investigate every bug report. We try and catch problems early. And we want to get relevant improvements and fixes into the software as soon as possible. This is what drove our implementation of in-app updates. And this is what drives our efforts in keeping Starry Night Beta going, even after SN7 has been released - because updates need testing too!

There are many arms to the beta (sometimes also referred to as "nightly") builds of SN7 and the systems used to get changes as quickly as possible into the hands of our testers. Here they are in their raw, technical jargon. All of the following tasks are fully automated, set in a sequence, and alert the developers of any problems encountered along the way.

Mac: checkout from the relevant branch of SN7 git repository
Windows: checkout from the relevant branch of SN7 git repository
Mac: build SN7
Mac: checkout from the relevant branch of Sky Data git repository
Mac: pool all the data for Pro Plus into the new Sky Data folder
Mac: add in the Manual and User guide to the final application
Mac: checkout from the relevant branch of AllSkyImage git repository
Mac: add all the AllSkyImage data that needs to be bundled into Pro Plus final app
Mac: build the latest Mac telescope plugin and add it to the application
Mac: codesign the final application bundle (to satisfy Gatekeeper)
Mac: store the freshly built app until relevant build number in our application storage
Mac: construct a pkg installer of the application
Mac: build a DMG disk image for the pkg installer to make the installation clearer and easier
Fire up a very large AWS EC2 instance that is used to quickly compress Windows installation data
Mac: rsync final application to the EC2 instance
Mac: create a drag-install DMG on Linux using genisoimage and libdmg-hfsplus
Mac: push built dmg to S3 for (full) updates
Mac: push the new build number to the relevant branch of SN7 git repository
Mac: package and push the new symbols file of our application to HockeyApp - the service we use to receive crash reports
Mac: read the relevant build release notes, if any, from our SN7 git repository, otherwise use current date stamp as a comment for the beta release
Mac: add details about the freshly built application to our release database (size, build number, version number, location of DMG file)
Mac: retrieve all active beta builds from the database
Mac: go through each previous version and create a delta update between the current and specific previous version (using BinaryDelta from Sparkle project)
Mac: for each delta update created, make an entry in our database with information about the update, size, versions involved
Mac: push each delta update to S3 for distribution
Windows: start a Windows instance on Parallels (we automate Windows building by invoking tasks from Mac to our virtual machine on Parallels)
Windows: build and sign SN7
Windows: copy Manual and User guide to the app
Windows: create a new full build of Windows for local app archive. Use already built Sky Data from our constructed Mac app.
Windows: build and add in relevant Windows plugins (telescope, focuser, MaxIm DL)
Windows: format release notes so that they can be used in automated xml-powered update building process with wyBuild
Windows: prepare specifications for delta update building by writing out an xml file with the latest information
Windows: build all delta updates using wyBuild. Let wyBuild upload all the data to our S3 server for distribution.
Windows: take the newly created wyUpdate.exe and client.wyc and put them into a folder used to assemble our Windows installer
Windows: push the additions to our wyBuild project and new wyUpdate.exe, client.wyc and wyserver.wys files to the relevant branch of SN7 git repository
Windows: launch and build our installer by invoking Advanced Installer on our aip project file (this only builds the minimal setup)
Windows: package and push the new symbols file of our application to HockeyApp
Windows: push the built installer to our EC2 instance
Windows: launch 7z compression separately on AllSkyImage and Sky Data on EC2 instance
Windows: using winrar on Linux, compress the installer, additional data 7z archives and custom modified SFX file header into Self-Extracting single exe
Windows: sign the installer exe with signcode (from github.com/mono project) on Linux to conform with Microsoft Authenticode
Windows: copy the built single-exe installer to S3 for distribution
Windows: shutdown the Parallels VM
Shutdown the EC2 instance
Mac: copy the built pkg-based dmg installer to S3 for distribution (this is currently the slowest part of the process which is why it is last)

Finally, on the distribution chain, we keep things separate between public releases and Starry Night Beta. Applications built for public release are essentially different builds (which is why beta versions and release versions will never again have the same build numbers). Beta builds only get beta updates. The same rule applies for public releases. On Mac, our appcast server reads different entries from our release database (to which information is added during the above build steps). On Windows we keep separate wyserver.wys and wyBuild projects - one for public releases and one for beta.

And this is just the nightly automated script for a single application (Pro Plus 7)! There is still a lot to be done to improve this system, parallelize some processes and implement failover solutions for others. We are currently using Parallels VM in order to easily script everything directly from one source. However, a Parallels VM could be replaced with another EC2 (Windows) instance. We want to keep the nightly builds no matter what and to make them faster and more robust! Even beta needs testing and development (a beta for Starry Night Beta?).

Until next time! Clear skies!