Deploying updated libraries (was Replacing Omnis VCS with SVN/Git type repository)

Tue Jan 2 09:52:08 EST 2018

Hi Alex, Doug,

Thanks for the positive feedback.
Regarding the launcher, I am aware it is not very sophisticated but 
figured it couldn't hurt to have another library on Git.  And as our 
publicly deployed apps are JS client server based, we have complete 
control internally for upgrades.  So I just needed something simple for 
my own use, basically.  I also use it in development mode to quickly 
reopen what I am working on after Omnis crashes :-( , and my WIP 
libraries are never in my application directory.

Doug,
Yes, it was quite a chore, taken me most of my time since EurOmnis to 
complete it.  I was aware of your BootLibrary and Python restarter, we 
are using the latter ourselves but purely to restart our thin client 
server apps on a regular basis.

Alex,
How are you assigning (and reading) the build number of your libraries?  
I have been thinking along similar lines, perhaps, in our case, using 
the SVN revision.

We have done some work on automatic updates along the lines of Doug's 
solution but nothing fully realised as yet.

Regards,
Graham

On 02/01/2018 14:19, Doug Easterbrook wrote:
> hi Graham…
>
> wooo hooo .. and so exciting.    Thanks for sharing the news.
>
> I’ll bet it was a bit of a chore to break the libraries apart into 
> reasonable constituent components. Its one of the tenants of sharing 
> code —   smaller, callable, includable functions are sort of the norm 
> for this kind of world.      We’ll be doing the same as we break apart 
> our monolithic large single library application;   After all, there 
> are things we use it in that we have open sourced … and so we could 
> eat our own dog food.    I”m working to get there.
>
>
>
> ok,  no as far as libraries for start-up go, I think we are like alex.
>
>
> we have one library called ‘BootLibrary.lbs’  It opens the libraries 
> within a sub folder called TM in a specified order.  so our structure 
> within ‘firstRunInstall’ is
>
>
> Startup
> — BootLibrary.lbs
> — TM (folder)
> ———  Theatre.lbs
> ———  spell check libraries
>
>
>
>
> the FUNCTIONS within BootLibrary.lbs is twofold
>
> 1) on startup the first time, it opens the libraries in a specific 
> order.  Yes it is in Code, but as Alex points out, its a short step to 
> put it into a configuration list (I like this idea).
>
>
>
> 2) Our updates come from the customers’ database since we’ve made that 
> the central repository of code. Specifically:
>
> - the new library (s) are put into a field in the database in Compress 
> format using some method.    We have a daemon that watches out site, 
> downloads, inserts into customer database ready to deploy
>
> - a version number is incremented
>
>
> and when the customer logs in, we look at the version file.   If there 
> is a newer version of Theatre.lbs, we read it, uncompress it, stick it 
> into the TM folder and then call it.  This allows different versions 
> of Studio runtimes to open the file and launch the application.     My 
> holiday testing was making Studio 5 and studio 8 co-exist and 
> auto-update the library file.
>
>
>
> this approach easily be expanded to have a compressed set of multiple 
> libraries … so the BootLibrary could decompress and install all versions.
>
>
>
>
> 3) and we have a restarter library (python) that we’ve given away in 
> the past that you can call .. and it will nicely help omnis shutdown, 
> including the boot library, and then it will restart the boot library 
> with the user’s login credentials .. after decompressing the updated 
> libraries.
>
>
> makes a very nice ‘update me while I’m idle and running Studio’ 
> capability.
>
>
>
> anyway..   a topic for next euromnis for sure — updating on the run.
>
>
>
>
> thats what we did.   happy to give away our library as well.
>
> and again…  where this whole open source stuff is going.   we all have 
> needs for these little helpers .. and we can write once, deploy many 
> places and save ourselves a bunch of time.
>
>
> woo hoo.       looking forward to mailchimp !!!
>
>
> *
> **Doug Easterbrook*
> *Arts Management Systems Ltd.*
> mailto:doug at artsman.com
> http://www.artsman.com
> Phone (403) 650-1978
>
>
>
> see you at the third annual users conference
> https://tickets.proctors.org/TheatreManager/95/online?performance=29086
>
>> On Jan 2, 2018, at 6:38 AM, Alex Clay <aclay at mac.com 
>> <mailto:aclay at mac.com>> wrote:
>>
>> Hi Graham.
>>
>> This is excellent to hear! We have a similar transition planned to 
>> break our associated classes out to separate libraries so we can 
>> build our Studio projects out of Git. It's good to know we won't be 
>> the first down that path. :)
>>
>> We built an app_loader.lbs which accomplishes a similar task as your 
>> launcher.lbs, but with a slightly different approach. Extra libraries 
>> are always located in a lib/ directory in the same location as 
>> startup/. We load these libraries first with the !!! prefix to avoid 
>> and startup tasks, then load the final library from startup/.
>>
>> The other difference is that we track an application build number. We 
>> use this to compare the libraries inside the application's 
>> firstruninstall/ with the user's %LOCALAPPDATA% or 
>> ~/Library/Application Support folder and, if different, update the 
>> user-writable libraries with the "official" copies from the application.
>>
>> How are you handling updating the application libraries and the 
>> user-writeable ones? I'd love to converge our solutions and provide 
>> something for the community. I feel managing these two copies of 
>> libraries is something that Omnis itself doesn't solve well but 
>> requires a bullet-proof solution.
>>
>> Alex
>>
>>
>>> On Jan 2, 2018, at 08:23, Graham Stevens <graham.stevens at gmail.com 
>>> <mailto:graham.stevens at gmail.com>> wrote:
>>>
>>> Hi All,
>>>
>>> Here at Platform Independent Systems, we have a number of Omnis 
>>> applications, covering desktop, JS client and thin client.  Until 
>>> now, these libraries have been versioned using the Omnis VCS.  But 
>>> with the advent of the new JSON representation of libraries and the 
>>> desire within the community to open source more Omnis projects, I 
>>> have been working on converting all our libraries.  We have now 
>>> completed the work so we wanted to share our experiences with the 
>>> Omnis community in the hope that it may encourage others to make the 
>>> change.
>>>
>>> The primary issue we had with the change was the existence of over 
>>> 200 classes which were shared by all our libraries.  The VCS had an 
>>> "infrastructure" project that was used purely as a repository for 
>>> the common classes and never built as an application - as all its 
>>> classes were shared with all other projects and consequently 
>>> included in the build of the other libraries.  We like this feature 
>>> of the VCS and used it extensively although the VCS interface is a 
>>> bit buggy and we had to manually adjust internal VCS records to keep 
>>> things in pristine condition.
>>>
>>> After posing the question on this list as to how to handle this 
>>> situation with a Git type repository, and considering everyone’s 
>>> input on the topic, we made the decision to keep these common 
>>> classes in a library which would be a required constituent of any 
>>> application.  On top of this we have a couple of applications that 
>>> consist of a desktop/JS client that communicates with a RESTful 
>>> server and these also had common classes particular to the 
>>> application in question.  So there are three "common" libraries, one 
>>> of which is common to all of the applications.  These "common" 
>>> libraries exist as an integral part of an application but do not 
>>> have startup tasks, they are purely class repositories.
>>>
>>> Deciding on this structure was the easy bit, the time consuming part 
>>> was the updating of the code in each application library to prefix 
>>> each reference to a class with the "infra." or "appcommon." library 
>>> in which it now resides.
>>>
>>> Having made all these changes, we then had to export all the 
>>> libraries to their JSON representation.  This presented us with 
>>> another few issues peculiar to the JSON exports.  To name a couple 
>>> that came up:
>>>
>>> -  Failing to have one or other of the required libraries open 
>>> before export
>>> -  References in legacy code to non-existent variables, ie. #???
>>> -  Duplicate names for objects on window and remote form classes
>>>
>>> Once we had fixed all the issues and successfully placed all our 
>>> libraries in our SVN repository (we already had a SubVersion server 
>>> for another project we had worked on), we had to come up with a 
>>> replacement for the VCS "Build Project" command to deploy our apps. 
>>> This is the point where Alex Clay's presentations at EurOmnis and 
>>> his libraries became invaluable and we have to offer him a huge 
>>> thank you for open sourcing them: omniscli.lbs 
>>> (https://github.com/suransys/omniscli) and ci.lbs 
>>> (https://github.com/omnis-ci/ci.lbs).  These two libraries now 
>>> enable us to script the building of our applications ready for 
>>> deployment.  The omniscli library accepts command line inputs to 
>>> control the ci library which builds the libraries from the JSON 
>>> source code.  I submitted a small enhancement to the ci.lbs 
>>> repository to extend the functionality to accept an optional list of 
>>> the dependant libraries' paths (infra and appcommon) to be opened 
>>> before building the application library.
>>>
>>> Not being particularly adept at shell scripting, I have created a 
>>> small Python script which accepts the name of the library to be 
>>> built.  It then calls "svn export" to copy the source code from our 
>>> SVN server to a local directory and then passes the path via 
>>> omniscli to ci.lbs to build the library. And this all takes no more 
>>> time than opening the old VCS and building a library.
>>>
>>> The final piece of the puzzle is the opening of our application 
>>> library.  We used to place our library in the Omnis startup folder 
>>> but now that we have one (or two) required libs to open before the 
>>> main library this isn't always going to work.  Libraries in the 
>>> startup folder seem to be opened in alphabetical order which isn't 
>>> always ideal.  So to circumvent this, I created a launcher.lbs that 
>>> is placed in the startup folder. This library's sole purpose is to 
>>> read an initialisation file containing the paths to the other 
>>> libraries, open them in the order they appear in the file and then 
>>> close itself.  The added advantage is that the libraries can reside 
>>> anywhere you like.  If anyone thinks it might be useful, we have put 
>>> it on Github here: https://github.com/PISL/omnis-launcher
>>>
>>> Now that I have successfully converted our libraries to JSON, we 
>>> will be releasing some of our code to the community on Git. (Doug, 
>>> this will include the MailChimp stuff!)
>>>
>>> Apologies for the length of this post but I hope it may be of 
>>> interest to some of you.
>>>
>>> And I wish everyone all the best for 2018.
>>>
>>> Regards,
>>> Graham
>>>
>>>
>>> _____________________________________________________________
>>> Manage your list subscriptions at http://lists.omnis-dev.com
>>
>> _____________________________________________________________
>> Manage your list subscriptions at http://lists.omnis-dev.com
>