AW: remove duplicates in a list

[Udo Sonnabend]

Hi, if the lists are big there are better approaches in O(m+n), see:

http://articles.leetcode.com/here-is-phone-screening-question-from/

Finding intersection of two sorted arrays – LeetCode<http://articles.leetcode.com/here-is-phone-screening-question-from/>
articles.leetcode.com
Find the intersection of two sorted arrays. Let’s called array1 as A and array2 as B, each with size m and n. The obvious brute-force solution is to scan through ...



Regards, Udo.

________________________________
Von: omnisdev-en <omnisdev-en-bounces at lists.omnis-dev.com> im Auftrag von Stephane Pras <stephane.pras at noos.fr>
Gesendet: Samstag, 5. November 2016 20:31:16
An: OmnisDev List - English
Betreff: Re: remove duplicates in a list

Grant,

I¹m not quite sure I understand your problem, but if you want to end up with
"a new list with only the unique elements², I would begin with merging them.

For chasing multi-duplicates based on a single field (let¹s say SKU), in a
single list, I use the following method:

Set sort field SKU
Find first  (Use sort)
Repeat ;; to browse all lines in the list
> Calculate iVar as SKU
> Next
> If SKU=iVar     ;; find all the lines with the same SKU value as the first one
> <<< delete or do whatever you like >>>
> Next
> End If
Until flag false

If you do not mean lines with the same SKU, but with the same value in each
field, you need a combined search with .AND.

I hope this will help,

Stephane

From:  omnisdev-en <omnisdev-en-bounces at lists.omnis-dev.com> on behalf of
Grant Thiessen bookitinc <bookit at bookitinc.com>
Reply-To:  OmnisDev List - English <omnisdev-en at lists.omnis-dev.com>
Date:  Saturday, November 5, 2016 at 3:33 PM
To:  OmnisDev List - English <omnisdev-en at lists.omnis-dev.com>
Subject:  Re: remove duplicates in a list

not sure that would work.  If I remove duplicates as before: I am removing
duplicates based on an SKU.  $removeduplicates removes 1 of the duplicated
lines, leaving the second duplicated line intact.  The process as I am using
it now adds additional lines to the list if they were not already present.

I think I want to eliminate both lines which are duplicates of each other,
which your solution does not seem to address, so that I am left with only
the lines which are in only one of the lists.

Or do I misunderstand this?

Grant

>  On 11-05-2016, at 9:04 AM, Doug Easterbrook <doug at artsman.com> wrote:
>
>  simple.
>
>  add a column to each list called Œoriginal_list¹
>  make that value Œ1¹ in list 1
>  make that value Œ2¹ in list 2
>  combine lists
>  remove duplicates as before.
>
>
>  now you know
>  a) unique values across both lists
>  b) which list it was originally in
>
>  its a variation of the old Œbalance line¹ algorithm used for sequential file
> updates thats been around for decades.
>
>
>
>  Doug Easterbrook
>  Arts Management Systems Ltd.
>  mailto:doug at artsman.com
>  http://www.artsman.com
>  Phone (403) 650-1978
>
>>  On Nov 5, 2016, at 7:45 AM, Grant Thiessen bookitinc <bookit at bookitinc.com>
>> wrote:
>>
>>  I have long used the $removeduplicates command to remove duplicates from a
>> single list, but now I need to do more than that.
>>
>>  I have two lists, each of which contains 10,000 plus lines.  I need to
>> compare them, and eliminate all the duplicated lines, to arrive at a new list
>> with only the unique elements (of which I know I am looking for about 300
>> lines).
>>
>>  Has anyone done this? or have an idea about a simple way to do this?
>>
>>  Grant
>>
>>
>>  _____________________________________________________________
>>  Manage your list subscriptions at http://lists.omnis-dev.com
>
>  _____________________________________________________________
>  Manage your list subscriptions at http://lists.omnis-dev.com


_____________________________________________________________
Manage your list subscriptions at http://lists.omnis-dev.com



_____________________________________________________________
Manage your list subscriptions at http://lists.omnis-dev.com