[Feature request] Optimization steps

[ammnt]

Hello,

please, consider about implement these optimization steps:

• remove empty lines;
• remove leading and trailing spaces;
• sort alphabetically;
• remove ||www.example.com rules or replacing them to ||example.com^;

Thank you.
Cheers☺️

[ammnt]

• Convert list to lower case;
• sort by TLD.

[scripthunter7]

Empty lines and trim are done: #33

@ameshkov In your opinion what is the correct sorting strategy if a rule has one or more comments? For example:

Input:

! Random comment 1, since it is followed by an empty line, it does not belong to any of the rules

b.com
! Comment for a.com
a.com

! Random comment 2, since it is followed by an empty line, it does not belong to any of the rules

d.com
c.com

Possible output:

! Random comment, since it is followed by an empty line, it does not belong to any of the rules

! Comment for a.com
a.com
b.com

! Random comment 2, since it is followed by an empty line, it does not belong to any of the rules

c.com
d.com

I've done something similar before, here's the link:
https://github.com/hufilter/hufilter-dev/tree/master/assets/scripts/orderer

[ameshkov]

@scripthunter7 Hmm, that's a good question.

Your approach seems very reasonable to me.

[scripthunter7]

@ameshkov As my free time allows, I'd like to further develop the linked script and convert it to an NPM package. After I'm done with it, the package can be easily integrated to Hostlist Compiler as a transformation such as SortRulesDesc / SortRulesAsc / SortRules or anything else.

It is not closely related to this topic, but I will describe what I mean by further development. The rules could be grouped according to their type, then the rules within that group can then be sorted again. For example:

domain3.org##.selector3
domain2.org##.selector2
#.selector5
#.selector4
domain1.org##.selector1

! Comment
#.selector7

to:

#.selector4
#.selector5
! Comment
#.selector7
domain1.org##.selector1
domain2.org##.selector2
domain3.org##.selector3

The configuration for this sorting script can be something like this:

{
    "groupOrder": [
        "general_hide",
        "specific_hide",
        etc.
    ]
}

It is similar to the current script anyway, but it does not automatically detect groups yet.

However, it is also necessary to clarify key issues such as the proper handling of comments (headers / sections / conditions etc).

[ameshkov]

However, it is also necessary to clarify key issues such as the proper handling of comments (headers / sections / conditions etc).

Yeah, pre-compiler directives should retain the order and work like a comment with a following empty line.

Also, there are hints that should be also considered.