Handling Diacritical Marked Text

[Ben Butler]

Das,

It will save what your program tells it to save asa distinct character. But
for the characters where there is human ambiguity such as single and double
striaght vs curley quotes it will save what you tell it to save and import
that into a utf8 table.  But we are telling it to save different things
because they are different characters.

It is neither a utf8 or db problem it is a question about what happens in a
db querry when the user supplied quotes dont match the imported data....

Of the top of my head I would be tempted to have a display field and a
stripped field. You strip all fancy qulotes and use the striped db field in
the where clause but in display use the none stripped field.

Others may have bettter ideas for cross language and cross platform.

PC and Linux cant even agree what caraiage return / line feed vs new line
look like in what is saved into the file data - not sure what mac does but
likely the Unix way.

It is a diverse source input problem.

Ben

On Tue, 18 Aug 2020, 23:55 Das Goravani, <goravanis at gmail.com> wrote:

>
> I kinda need to know or confirm this:
>
> I’m about to test if a UTF8 doc saved that way from text editor settings
> cures my problem upon import to Omnis
>
> But the thing is I saved from this same program and it defaults to UTF8 so
> that is what I used the first go round
>
> And it came out wrong
>
> So is it SQLite?
>
> Is it possibly Pragma set NOT on UTF8?
>
> Can I check if it’s UTF8?
>
> Would you guess Omnis would create the SQLite database the wrong way?
>
> Because obviously the text editor gets it.
>
> UTF8 is the way to go to handle the good stuff
>
> It’s puzzling in other words WHERE is the Problem I had?
>
> The states and countries came out with wrong stuff but they were saved
> UTF8 the first time.. problem is in Omnis or SQLite
>
> Can you tell me which and how to fix it?
>
> Pragma setting I get it may be I want to know if I can check that out.. or
> how if I switch it does that affect data I have in SQLite now that was put
> in perhaps under a different encoding.. how do you set the encoding Omnis
> is adhering to or do you?
>
>
> _____________________________________________________________
> Manage your list subscriptions at http://lists.omnis-dev.com
> Start a new message -> mailto:omnisdev-en at lists.omnis-dev.com
>