The App Server and the CRB

Sun Jun 21 10:00:23 EDT 2020

I should state the source of my understanding and the follow on experience we had.

When we were building our web sales processes in studio some 15 years ago, we were migrating from *similar* code in Classic.

In classic, there was no built in capability for web services.  That is to say, until we managed to find Kelly’s super valuable TCP talk. —   that made web services a viable option.       However, Kelly’s TCP/talk was single threaded —  you only ever got one request at a time and others waited (nicely as it turns out in a queue in TCP/Talk)

Consequently, in Omnis classic we never worried about using MOFFS (memory only file formats), hash vars, lists, you name it.

Move forward to Studio where there is now the concept of a ‘remote task’.     Michael Monschau joined our conversion team for about a year and he was the re-architect of our web sales processes using remote tasks.     While Michael did the underlying architectural code, much of the actual web response code was copied from classic (since it worked).

Michael told us that each new instance of the remote task HAD ITS OWN COPY of variables.   We believed Michael as he was, at that time, a ‘former’ but very recently ‘former' core omnis engineer with lots of knowledge in the inner workings of Studio.

on top of that, we had relied heavily on MOFFS to fill web pages.    Specifically, there is code in our HTML web page templates that used a concept from Kelly’s suggestion that we have  <field> tags that surround omnis variables.   so if the omnis MOFF variable is something like  PS_TICKET_QUANTITY, we would stick that in a web page template looking something like:

<html>
<head>
</head>
<body>
This is a variable containing quantity  times 10 <field> PS_TICKET_QUANTITY*10</field>
</body>
</html>

that means each and every response to a web request loaded up MOFF fields so that we could merge them into an HTML template by looking for the <field> tags and using an eval() on the text between the <field> and </field> tags

Michael saw how we had done the filling of web pages in the classic version and had no qualms about using MOFFS.

For the last 10 years, using Studio 4 & 5, we ran web services with multiUser licences, using MOFF fields —and we never had leakage between memory only file formats that I recall.  Our web pages have been essentially the same for nearly 20 years and nobody ever bought the wrong inventory from our web sites when we use MOFF’s.

so, coupled with
- Michael’s knowledge of remote tasks & variable instantiation,
- our usage of MOFF’s in web pages and
- years of history, of no cross talk

 I’d be pretty confident that we were properly down the path of how to do it.

would I design web serices like this any more?  short answer is no.  and here’s why.

The reasons that we no longer use a single copy of omnis on a machine to run multiple listeners (even with a mutli use licence) is that we also discovered limitations that affected our scaleablility.

Specifically Omnis does everything in the remote task on thread zero - Studio Code that you write in a procedure is not multi threaded in anyway and never can be.

The BEST way to take advantage of omnis on machines with multiple cores is to start MANY copies of omnis and have something (like nginx load balancer type processes) orchestrate delivery to each running copy of omnis.    In this way, we could get 16 or so copies of omnis running on the same machine and scaling up to use the most of the available CPU resources.  (a 16 user licence, or 16 individual licences, one for each copy of omnis are effectively the same).

Omnis, with studio 10.1, offers a feature that does this for you - it scales up multiple copies of omnis.  Again omnis is not multi threaded, but it sure can do multi-process.   To explain the difference:

multi-thread = single copy of a program starts up multiple threads and handles all the difficult inter process communication to start, stop and coordinate within itself.
multi-process=  forget about having threads… just make multiple complete instances of yourself that can’t talk back and forward.

multi-process architecture is how Postgres and a number of  high performing services do things.   and Omnis can too.   This is fantastic.

This leads to the conclusion of MOFF’s and cross talk discussion and what is probably the better architecture for building scaleable web services.

in a single copy of omnis, that is using remote tasks to serve web pages, you are, from my experience, immune from cross talk between remote tasks as each remote task makes its own copies of hash and MOFF variables in their own CRB records.

If there is still concern that you might have such crosstalk, avoid using a single copy of omnis.    Instead, use the multi-process approach to omnis servers and start up multiple copies of omnis.  it is 100% guaranteed that you will NEVER have cross talk with this approach — and you’ll get to use far more of your machines as servers.

We do not use Omnis’s load balancing (we wrote our own).   I *think* (meaning I would like to stand corrected and will happily bow to the knowledge of others that have used it) that omnis calls this Server Load Sharing as part of their APP server.    It seems that there is a program called OmnisLSP.  (omnis load sharing program??) that will handle distribution requests amongst multiple copies of omnis app servers.     I hope somebody who has used this will jump in and point people in the proper multi-process direction.

thats what I know as of this morning.

Doug Easterbrook
Arts Management Systems Ltd.
mailto:doug at artsman.com
http://www.artsman.com
Phone (403) 650-1978

> On June 20, 2020, at 8:31 PM, malkishtini <malkishtini at gmail.com> wrote:
> 
> Hi,
> 
> This is actually an interesting subject that is still puzzling to me.
> I just finished the work on adding an API to print an Omnis report in PDF format in an application that was developed in O7 and converted to studio as is (without reengineering), so the code still uses memory only file classes and CRB to build the lists that are used in the report.
> All the web apps and the ultra-thin clients that we have developed so far in Omnis are using instance and local vars..No web app was built before using global vars.
> 
> So since I was not sure how global vars/CRBs will work with remotes tasks when multiple concurrent requests are made to the API with different params and how accurate the result will be, I sent a question to Omnis support with samples of my code(how the calls are done from the remote task to the other methods that are in different classes) and the way the lists are built using variable from different methods, and they answered me, the way I understood their answer was that it is better to issue one request at a time since we can't trust the variables that is loaded in the CRB at the time the code of the method is executed.
> Unless I misunderstood their response, that's my understanding to how global vars are handled when running in remote tasks.
> I hope I'm wrong in my understanding, since that will affect the way the ORA can be consumed in our case.
> 
> The other thing that is not clear to me yet is the multi-threading in Omnis, if multiple requests are sent to the server one after the other from a single source, can Omnis handle all of them at the same time or there will be a delay, cause Omnis has to finish one request then process the next one? i.e. does Omnis App server (version 10) on Windows support multi-threading or not?
> 
> Thank you,
> Mayada
> 
> -----Original Message-----
> From: omnisdev-en [mailto:omnisdev-en-bounces at lists.omnis-dev.com] On Behalf Of Doug Easterbrook
> Sent: Saturday, June 20, 2020 8:05 PM
> To: OmnisDev List - English
> Subject: Re: The App Server and the CRB
> 
> hi das:
> 
> there is an interesting thing about serving web sites from omnis.     Each of our connections instantiates a remote task.  so I’m assuming what you are doing is ths.
> 
> each remote task connection gets its own set of variables and they cannot be seen across threads.
> 
> so, if you have two people connected, each one has its own CRB variables, hash variables, etc and you don’t have to worry about cross talk.
> 
> 
> one word of warning…
> 
> it is possible to create a reference when you start a remote task that points back to the main task.   we have done this to share database connections and a few other key bits of data that are somewhat static and don’t need to be read each time.
> 
> if you do that.. and you’ve built some lists in your main omnis task, make sure you do not manipulate those lists or variables
> 
> instead, COPY them to a local copy and manipulate the local copy.  that way you wont have some surprizing results
> 
> 
> 
> eg.
> 
> in the main task
> 
> creates a list called ‘Months’ that contain 12 rows with values Jan to dec.
> 
> 
> 
> in the Remote task
> 
> this cannot see the list called ‘Months’.       but you can set a reference back to the main task to get the list
> 
> reg.   set reference Localtaskref to $tasks.Main.$ref
> 
> calculate localMonths as taskref.$getMonthList
> 
> 
> 
> now you have copied the Months list to a variable called localMonths.  and you are safe
> 
> 
> if, instead, you manipulated Months and set the value to the row for dec ….    another remote task could change it to august on you… and you wouldn't know.
> 
> 
> 
> 
> 
> 
> for a recap:
> 
> 1) crb vars and #hash variables are distinct for each remote task
> 
> 2) you can refer to common variables via the main task to get constants .. but don’t manipulate that data in the main task from the remote task.  copy the data instead.
> 
> 
> 
> hope that helps.
> 
> 
> 
> 
> 
> Doug Easterbrook
> Arts Management Systems Ltd.
> mailto:doug at artsman.com
> http://www.artsman.com
> Phone (403) 650-1978
> 
>> On June 20, 2020, at 2:35 PM, Das Goravani <das at Goravani.com> wrote:
>> 
>> 
>> Hello
>> 
>> I’m trying to understand the difference between the CRB and using rows to hold data in the App Server.
>> 
>> Say you have a chunk of code, big, that does one thing.. in my case an astrological chart and readings.. then another big chunk of code and reports that creates a PDF output.  You want these online.. using the App Server.. so that people can get astrological reports.
>> 
>> Using the App Server.. say one persons code is running.. and their time slice ends.. the next person in the que starts to run and overwrites the values in the CRB with new ones.. bad.. I’m led to believe presently that is what happens.. that the CRB is shared between users..
>> 
>> And people said therefore you have to use ROWS and reference them rather than fields from file formats.. but rows and lists are ALSO in the CRB and subject to being written over aren’t they?
>> 
>> Or is it that what’s meant is to use an object for all your code.. and variables.. and then OBJECTS INSTANTIATE FOR EACH USER????
>> 
>> Do they?
>> 
>> How do you get your rows and lists to NOT be overwritten just as the CRB is.. ???
>> 
>> If you widen the time slice so that there’s enough time for it to finish the part that is in the CRB do you think that is viable as a way?
>> 
>> It would be a really huge task to convert all my code in those two chunks to being all rows and lists holding data and references to them in all code..
>> 
>> I’m thinking of putting my astrology readings online.. thinking of using the app server.. if I kick off users that hasn’t budged in 5 minutes maybe I can keep the number of users down.. and affordable.. the unlimited users price tag is a bit high for me at like 14 K … wish I could do it.. then I wouldn’t have to kick off users..
>> 
>> This note is about how does rows do better than the CRB in terms of keeping users data separate on the app server.. that is really the question..
>> 
>> Das
>> _____________________________________________________________
>> Manage your list subscriptions at http://lists.omnis-dev.com Start a
>> new message -> mailto:omnisdev-en at lists.omnis-dev.com
> 
> 
> _____________________________________________________________
> Manage your list subscriptions at http://lists.omnis-dev.com
> Start a new message -> mailto:omnisdev-en at lists.omnis-dev.com