Desktop apps and Web - what makes sense?

Doug Easterbrook doug at artsman.com
Thu Aug 12 16:31:00 UTC 2021


hi Greg:

We developed our REST api a while back.  Then, when we ware Euromnis about 3/4 years ago, I sat in on Jason’s presentation on making rest api’s in omnis.

To my recollection, it seemed very simple.  effectively, something like
- open the port
- create a task that would listen
- name the method in the task as the API end points
- then just use them

eg, if an end point is  $client you would access it like.     

www.myserver.com/client -> to see a list of clients
www.myserver.com/client/10 -> to see client 10


I am probably totally over simplifying it .. but it struck me that the way the rest api could be made was very very much like what we  researched and built in python.



This is not a plug for python.   Instead its a plug to say

- get started.    Omnis has done a lot of work for you that helps you make your API standard.

- doing a rest api is a middle tier of the multi tier architecture and lets you get on your way

- you can always replace your middle tier later, if you so choose

- you can also mix technologies for the middle tier.   For example, if you find a particular endpoint is slow, you can
—— write that one endpoint in anther technology
— — move that endpoint to is own machine so that all client read access comes out of one machine and all client write access happens on another.

- and that allows scaleability.   you just need to have multiple servers and a front end that distributes load between servers (we like nginx far better than apache - apahe has failed us miserably in the past when it comes to stability and scale) 

- and that would also let you run your servers headless — if you ever wanted to cause use them in linux, rather than windows or mac.


thats been our general strategy to handle volume.   
1) if one server is not enough, use more on the same machine to use all your cores (omnis apache front end does load distribution)
2) only allow each server to do one task at a time (i.e. don’t ever think of multi threading, go multi process with omnis)
3) put servers on separate machine and load balance them when you get to that kind of load (apache does that too)
4) if things get even heavier, pick the sore spots and write in a different technology if need be

Note:  different technology could be the language the REST API is written in (eg omnis, python, c++, rust)
OR it could mean throwing some of the procedures into pgsql in the database.


we’ve done them all.  The advantage is that your app continues running while you swap out one very small, very testable component for another.

and to the point is to consider each endpoint in a rest api as a separate microservice rather than one whole monolithic application.

not, I did say separate microservice.   that DOES NOT mean break your application into many libraries.  I’m not a fan of that.  I much prefer one big one .. far easier to update/deploy a single one than many.

what I mean by a microserervice is that you built the each rest API endpoint to do one thing.   like get patron record,   like update patron record,   like find ticket.   like book ticket, like accept payment.

very very important to make things granular, all separated by their own endpoint on a URL.     it helps with future capability to separate servers, load balance servers, etc.   i.e. you decide which of many servers could handle the client rest endpoint (using apache dispatching) ….  even if each server has all the code.



so, microservices.      that also means your servers could use EXTERNAL microservices viar your rest endpoints.   if you wanted to lookup a postal code for an address, you could let your microservice forward on to canada post to get the answer.   just an example.


so.. back to comments:

> One thing that I am not sure about is the use of stored procedures.  I see the advantage of having multiple parties (including legacy Omnis desktop app) using them, however, the challenge is to maintain them in such a way that nothing gets broken. The debugging of stored procedures where you put biz logic might be another challenge, not a trivial thing.


true it is harder.   there is a command in pgsql called ‘raise’  
refer to: https://www.postgresql.org/docs/current/plpgsql-errors-and-messages.html

basically it lets you put out debugging info into the postgres log…. if you need to.   This is no different than a headless server where you have to write to a text file or log to see whats going on.


solution:  make small, testable routines if you want to have some stuff in pgsql.     to be clear, we DO NOT have all, nor do we have a significant portion of our business logic in postgres.

we have some.     and the places we chose to do it at the beginning is where we really needed performance for database I/O where many things happen on a single database call.     We do not use it for anything where we are simply reading or updating a large set of records.  thats all in omnis.


three advantage of SOME procedures in pgsql:
- when you need performance
- when you want to use triggers to update other data and maintain absolute integrity. 
- you are contemplating a complete rewrite of a function and it might be in another technology, so writer it in pgsql.. adapt omnis to use it.  that will give you a prototype .. then use another technology as the interface.

but DO NOT (please do not) take that as a blanket statement to write everything in pgsql.    perhaps 0.5% of our code is in pgsql.   its just an option.

> 
> I was thinking that Omnis, since the desktop app is the starting point, might be the best place to keep biz logic.  I was particularly thinking about Omnis REST service complementing Omnis desktop app

yes,  thats pretty much what this is about.  its the correct way to go forward.

> , having Web UI maybe done in non-Omnis technology, keeping the database free of any biz logic.  


yes.   pick your UI to be what helps your customers.

In our world, there are our customers … and then the customers of our customers.


everything that the customers (of our customers) use is all HTML5 forms using bootstrap.  it lets many tens of thousands of people use responsive pages to buy tickets with a customizable interface.   Using bootstrap is actually very easy.   customizing it is also easy if you want to re theme things very quickly.

https://help.theatremanager.com/web-page-documentation/bootstrap-live-customizer

you can introduce javascript if you want.   thats perfect for scaleability. we don’t tie up server resources while the customer makes up their mind what to buy.



fo our customers — we are predominantly using omnis client server .     We want to thin out the clinet so that we use omnis talking to our rest api.     In other words, we are not yet at 3 tier for everything… but we are slowly getting there while maintaining a full client server application.     Our customers don’t need to know how we are talking to things with our server infrastructure… they just see something always work...



don’t know if that helps.




Doug Easterbrook
Arts Management Systems Ltd.
mailto:doug at artsman.com
http://www.artsman.com
Phone (403) 650-1978

> On August 11, 2021, at 7:31 PM, Grzegorz (Greg) Pasternak <gpasternak at cogeco.ca> wrote:
> 
> Doug;
> 
> Finally, using LIFO approach I have managed to get back to you.  Yes, this topic would be great for late evening conversation but sadly we still have to wait for that.
> 
> One thing that I am not sure about is the use of stored procedures.  I see the advantage of having multiple parties (including legacy Omnis desktop app) using them, however, the challenge is to maintain them in such a way that nothing gets broken.  The debugging of stored procedures where you put biz logic might be another challenge, not a trivial thing.
> 
> I was thinking that Omnis, since the desktop app is the starting point, might be the best place to keep biz logic.  I was particularly thinking about Omnis REST service complementing Omnis desktop app, having Web UI maybe done in non-Omnis technology, keeping the database free of any biz logic.  
> 
> As usual nothing is simple, thanks again for all your valuable information, still pondering on it.
> 
> Greg
> 
> 
>> On Aug 10, 2021, at 7:04 PM, Doug Easterbrook <doug at artsman.com> wrote:
>> 
>> hi Greg.
>> 
>> almost .     
>> 
>> we put some things int stored procedures and triggers when we need some thing to be fast.  Lets see if I can cite some example for you.
>> 
>> SQL lets you update many records at one time.  eg update table set field1=10 could do many thousands of records if you don’t supply a where clause.   a stored procedure is not necessary for that.
>> 
>> on the other hand, if we are looking for available tickets and we want to suggest some for people taking into account proximity to other seats, not leaving single seats as part of an offering to the customer (single seats tend to stay that way, losing people revenue), taking into account seats that are held back until closer to the performance….
>> 
>> we use a stored procedure for that because simply offering one seat to you means we have to examine many around you, and if thats not right, we skip to a different part of the venue.   its very loop intensive.
>> 
>> doing it out on the internet — takes many individual sql calls and there is a lot of latency in transmitting the data and waiting for it to come back.   so a stored procedure does it in under a second.     time savings = huge.
>> 
>> 
>> in that case, part of the business logic is pulled out of omnis and moved closer to the database (stored proc) so its faster.
>> 
>> one final thing about store procedures… they are exactly on transaction for everything that is in it.  it either works or it doesn’t.    we don’t have to set up transaction boundaries.
>> 
>> 
>> 
>> the other place that we have put business logic is into python.   Why you might ask.
>> 
>> the python processes run on the same machine as postgres - so there is minimal latency.  its almost as good as a stored proc.        however, we have to worry about transaction boundaries and wrap multiple statements in.  begin(). commit() and do error checking with rollback().
>> 
>> 
>> we will put more business into python so that its close to the database and we will take those functions out of omnis.   why again?
>> 
>> because python plays a strong part in our 3 tier architecture.  meaning.   user(omnis)—> python (logic) —> postgres (data storage)
>> 
>> the central tier means we can have other services talk to python in ordr to update the data.
>> 
>> 
>> 
>> where does that lead us, technology wise.
>> 
>> 
>> ultimately, we have to remember that the internet is designed with breakage in mind.  connections die.   so if you write an app (in omnis) that assumes created a connection, uses it, and then throws it away in order to do something (like sell a ticket)… you start to win big
>> 
>> it means you no longer need a permanent database connection.   it means you can use an OW3 worker to send a query to the python server to update the database and end the connection.     you can do that many times simultaneously.
>> 
>> and when you do that, it gives you a robust, fault tolerant application that will fix itself up if an internet connection goes away (eg user closes laptop)  and then gets reestablished (use opens laptop)
>> 
>> 
>> to answer things directly:
>> 
>>> Omnis is really just a legacy desktop app that you keep and has nothing to do with the internet traffic, is this correct
>> 
>> not quite.  Omnis coordinates interface, error messages, allows data entry into fields, ensures data is complete, updates lists of data, provides reports .. gives local error messages, switched between tabs, … all the stuff it currently does.
>> 
>> however, the goal is to thin it out at the part where we would normally talk to the dam/table class to update the data … to call functions at the server (through a rest api or stored procedure — either/or)… and let the server handle the heavy work of ensuring the data is correct and putting into the database.
>> 
>> this minimizes data transmission where possible.
>> 
>> it allows other services (languages) to use the same rest api to get the same stuff done.
>> 
>> it allows users to write little micro services to tal fto their data without needing us.   example:  we have users that pull data and images out of their database to populate their main word press marketing web site (using the rest api)
>> 
>> 
>>> Further, are you moving the biz logic out of Omnis lbs into database to prevent code duplication?
>> 
>> 
>> 
>> yes. where its appropriate.     the goal is to do that 100%.     if all read and update logic is in the middle tier, it lets us change and optimize the business logic.  it also allows us to write automatic test cases to ensure data validity.
>> 
>> and given that more people writing code on a project tend to make it less successful, automated testing is one of the best ways of ensuring code works.    Code testing (shout out to Alex here for omnis tapi).. needs testing protocols/harnesses.      Server based coding with not interfaces, just business logic, tends to help a lot with automated testing processes.
>> 
>> 
>> thats lets u get more done with less.
>> 
>> 
>> hope that make sense.   man, this us a euromnis beer conversation.  
>> 
>> 
>> Doug Easterbrook
>> Arts Management Systems Ltd.
>> mailto:doug at artsman.com
>> http://www.artsman.com
>> Phone (403) 650-1978
>> 
>>> On August 10, 2021, at 1:42 PM, Grzegorz (Greg) Pasternak <gpasternak at cogeco.ca> wrote:
>>> 
>>> Doug;
>>> 
>>> If I understand correctly, in your implementation the biz logic resides to some degree in the database in stored procs, the Omnis is really just a legacy desktop app that you keep and has nothing to do with the internet traffic, is this correct?
>>> 
>>> Further, are you moving the biz logic out of Omnis lbs into database to prevent code duplication?
>>> 
>>> Greg
>>> 
>>> 
>>>> On Aug 10, 2021, at 4:29 PM, Doug Easterbrook <doug at artsman.com> wrote:
>>>> 
>>>> hi Greg.
>>>> 
>>>> for most apps, the ‘glue’ is the database/storage technology.      Everything we do derives from postgres.  even high volume fast access reddis files we use for caching - invariably start with postgres.       Postgres is the core.
>>>> 
>>>> around that … we use technologies that make success.   
>>>> 
>>>> eg, 
>>>> 
>>>> we use pgsql stored procedures when we weant some data manipulation to be really fast and close to the database
>>>> we use linux cause postgres is faster an more scaleable
>>>> we use CEPH on multiple machines to allow fault tolerant, cross machine, redundant storage of our data in postgres
>>>> we use ProxMox to manage the linux machines that manage our data
>>>> we use python because its really good at handling a lot of web requests simultaneously that allow people to get to the data
>>>> we are going to use reddis cache to handle sepmaphores to allow access to the database under high volume loads
>>>> we use omnis to handle back office data entry and reporting against the database.
>>>> we build rest api’s to let users and programmers get access to the database using 3 tier architecture
>>>> 
>>>> the glue is your data base.   it is the constant that needs to be guaranteed to be there
>>>> 
>>>> 
>>>> omnis is about 1/2 of our code base.  and its great at what it does on multiple platforms.  couldn’t do without it.      but the glue is the database.
>>>> 
>>>> 
>>>> Doug Easterbrook
>>>> Arts Management Systems Ltd.
>>>> mailto:doug at artsman.com
>>>> http://www.artsman.com
>>>> Phone (403) 650-1978
>>>> 
>>>>> On August 10, 2021, at 12:13 PM, Grzegorz (Greg) Pasternak <gpasternak at cogeco.ca> wrote:
>>>>> 
>>>>> Thank you for your comments.  
>>>>> 
>>>>> Funny you have mentioned Go, I worked with the programmer on the project that involved Go, it was supposed to be "the technology" but he changed his focus after completing this project and moved to different language.  Btw, I am agnostic about the actual tool set needed for web part, as this would be delegated to an expert (or aspiring expert).  All I am trying to find out is if Omnis is still considered to be somewhat relevant as the "glue" for different technologies used.
>>>>> 
>>>>> 
>>>>> 
>>>>>> On Aug 10, 2021, at 3:01 PM, ADJob <mats at adjob.se> wrote:
>>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>>> 1. Is Desktop Apps a thing of the past?
>>>>>> 
>>>>>> In my opinion - YES!
>>>>>> 
>>>>>>> 2. Is it making sense to keep existing Desktop Apps and develop complementary Web Apps?  
>>>>>> 
>>>>>> I think the migration from Desktop to Web will take some years, so you may keep the existing Desktop. But focus on the Web App.
>>>>>> 
>>>>>>> 3. Is it perhaps better to develop a Web App replacing Desktop App completely using non-Omnis technology?
>>>>>> 
>>>>>> I think that you will get the job done faster with Omnis Web. Maybe the magnitude of 5x. Non-Omnis technology consists of dozens of technologies you have to put together yourself.
>>>>>> The bottom line is that you have to choose between time and money. Omnis will cost you money and Open Source will cost you plenty of time.
>>>>>> 
>>>>>>> 4. Is there a place for Omnis to be a relevant part of Web App design?
>>>>>> 
>>>>>> About 50% of all developers are between 20-30 years according to StackOverflow. I asked the list if there are any youngsters on the list.
>>>>>> The answer was practically zero. So I guess that Omnis will lack new fresh blood, considering that most of the options are open source and free. So I have my doubts that Omnis Web is the right path. 
>>>>>> 
>>>>>>> 5. Is there any commercially successful implementation of the Web App using Omnis technology?  If so, which part is it?  (Javascript remote forms?  REST? Ultra thin client?)
>>>>>> 
>>>>>> I asked this question on the list a while ago and I got a dozen of answers. Not encouraging. I am investigating Go (Golang) that has about 2 millions developers today. https://go4webdev.org
>>>>>> But this path I should never consider if Omnis was more affordable.
>>>>>> 
>>>>>> /Mats
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> _____________________________________________________________
>>>>>> Manage your list subscriptions at http://lists.omnis-dev.com
>>>>>> Start a new message -> mailto:omnisdev-en at lists.omnis-dev.com 
>>>>> 
>>>>> _____________________________________________________________
>>>>> Manage your list subscriptions at http://lists.omnis-dev.com
>>>>> Start a new message -> mailto:omnisdev-en at lists.omnis-dev.com 
>>>> 
>>>> _____________________________________________________________
>>>> Manage your list subscriptions at http://lists.omnis-dev.com
>>>> Start a new message -> mailto:omnisdev-en at lists.omnis-dev.com 
>>> 
>>> _____________________________________________________________
>>> Manage your list subscriptions at http://lists.omnis-dev.com
>>> Start a new message -> mailto:omnisdev-en at lists.omnis-dev.com 
>> 
>> _____________________________________________________________
>> Manage your list subscriptions at http://lists.omnis-dev.com
>> Start a new message -> mailto:omnisdev-en at lists.omnis-dev.com 
> 
> _____________________________________________________________
> Manage your list subscriptions at http://lists.omnis-dev.com
> Start a new message -> mailto:omnisdev-en at lists.omnis-dev.com 



More information about the omnisdev-en mailing list