Importing 20 Million Records
dlr at futurechalk.com
Sat Jun 26 02:22:19 UTC 2021
If you want to do it in Omnis code you could do it with AppleScript. For example:
Begin text block
Text: tell application “Terminal" (Carriage return)
Text:delay 6 (Carriage return)
Text: do script “sqlite3" (Carriage return)
Text: do script ".open [database_name]" in window 1 (Carriage return)
Text: do script ".mode csv" in window 1 (Carriage return)
Text: do script ".import [import_file_name] members" in window 1 (Carriage return)
Text: do script "select count() from members;" in window 1 (Carriage return)
Text: end tell (Carriage return)
End text block
Get text block iScript
Quit method iErrCode
This will load the data from the csv into the SqLite database. It will probably take some tweaking to get it to work.
> On Jun 25, 2021, at 9:38 AM, Das Goravani <goravanis at gmail.com> wrote:
> Dear Friends,
> In OMnis I am familiar with importing, but never 20 million records.
> If you have a huge file with 20 million records in it, how do you import that?
> Omnis works with a list.. you import to a list.. Omnis can handle between 1 and 2 million rows.. from my experience
> Do you cut up the file.. do you get some utility for that?
> Because you can’t open such a file in any program like Word or Excel or Numbers or Pages
> So you’d need some utility meant for cutting up large files
> Or is there a way to tell Omnis to only import part, then next part, then next.. I’ve never seen that.
> Das Goravani
> Manage your list subscriptions at http://lists.omnis-dev.com
> Start a new message -> mailto:omnisdev-en at lists.omnis-dev.com
More information about the omnisdev-en