Saving Documents in PG vs File system

[Reg Paling]

Hi Mike,

My application stores documents in the file system.  On windows it works 
mostly without a problem but on macOS problems pop up regularly across 
the user base: permissions needing to be repaired, folder structures 
getting corrupted and so on.  I could not believe it during the first 
years after I took over the app, but the only conclusion based on my 
experience is that macOS is flakey.  Newer versions may be slightly less 
so but nonetheless all versions are flakey.  Some servers appear to be 
more vulnerable than others.  Attempts by IT support to rebuild and 
repair the disks don't work, but then the client buys a new server and 
the problems (sometimes) largely go away.

So I have been thinking of going in the opposite direction!  I like your 
idea of the temp and permanent databases.  I haven't done anything with 
S3 but my experience of other services from AWS is that they are indeed 
a beautiful thing as Andrew says.

Regards,
Reg



On 15/11/18 2:25 am, Michael Matthews via omnisdev-en wrote:
> Hello All,
>
> A question for those of you who store documents in your Omnis system.  We currently do, and we have a separate DB for the storage, and keep the metadata in the main database.
>
> However, in certain cases, the PG database size is now over 100gb when exported.  And 99% of that data is static, so a waste of time in backing it up etc every day.
>
> 1	I could split the database into 2 parts as well, 1 for new items, which will periodically move items to the 2nd archive database.
>
> 2	But I want to explore storing the files in the file system, and passing them to the user when asked for.  Can PostGres save files locally, and then read team and send them back to a user within a SQL call?
>
> 3	I also remember Marten Verhoven showing how he uses Symbolic links to files that would normally be hidden AND inaccessible unless given.
>
> So what do others do at this point with numbers of docs exceeding 100Gb now?
>
> Thank you
>
> Mike
>
>            <http://www.lineal.co.uk/>
>
>   <https://twitter.com/linealsoftware>  <https://www.facebook.com/linealsoftware?fref=ts&ref=br_tf>  <https://plus.google.com/112982815000763473226/about>  <https://www.linkedin.com/company/1859834?trk=tyah&trkInfo=clickedVertical%3Acompany%2Cidx%3A1-1-1%2CtarId%3A1437729256906%2Ctas%3Alineal%20s>		
> Mike Matthews
> omnis at lineal.co.uk <mailto:omnis at lineal.co.uk>
> Lineal Software Solutions
> 01271 375999
> Commercial House, The Strand
> Barnstaple, Devon, EX31 1EU
> www.lineal.co.uk <http://www.lineal.co.uk/>
>
>   <http://www.lineal.co.uk/windows-support/>  <http://www.lineal.co.uk/microsoft-office-365/>  <http://www.lineal.co.uk/wifi-networks/>  <http://www.lineal.co.uk/wifi-networks/>  <http://www.lineal.co.uk/it-systems/>  <http://www.lineal.co.uk/gamma-horizon-voip/>  <http://www.lineal.co.uk/kerio/>  <http://www.lineal.co.uk/apple-upgrades-repairs/>  <http://www.lineal.co.uk/eset-security-software/>  <http://www.lineal.co.uk/shoretel/>
>
>
>
> _____________________________________________________________
> Manage your list subscriptions at http://lists.omnis-dev.com
> Start a new message -> mailto:omnisdev-en at lists.omnis-dev.com