Limit thalamus region dirvecs to specific regions

[asoplata]

This replaces the region selection of what region direction-vectors are made for in thalamus with a regular expression. This was scientifically necessary for exclusion of the habenular and peripeduncular regions, and code-wise so that the thalamus region selection regex can be harmonized across all uses. For information on which regions were chosen and how this list was created, see the internal BBP Confluence page located at "Circuits > Mouse Thalamus > Atlas-based Whole-thalamus subregion selection". This regex has been built from the region list of the desired and present thalamus regions as of the "final" version of the hierarchy and annotation built by the Atlas pipeline, which is the output of the rule split_barrel_ccfv3_l23split.

[asoplata]

I don't know why some of the tests are failing, but this is a small change, and should be able to be integrated with no problems. This should not interfere with any tests specifically.

[lecriste]

I don't know why some of the tests are failing

https://github.com/BlueBrain/atlas-direction-vectors/actions/runs/8169293234/job/22333073941?pr=27#step:5:60

[arnaudon]

@asoplata , to understand the long list of subregions, you just don't want three of them, right? PP, LH and MH? I could be simpler to not have these, instead of the entire list of the ones you want, no?

[arnaudon]

It seems LH and MH are under EPI, with also has PIN, but this one you don't have in the list, and you don't mention as you don't want in the confluence page. I guess you don't want EPI, right?

[asoplata]

@arnaudon In theory, yes, it would be simpler to structure my query so that it gives me all of TH except for the subregions I do not want to include. However, as far as I can tell, I can't do this from using a simpler query.

As a test case, if I pass the following query to voxcell.RegionMap.find in a similar way to how query_region_mask does:

thalamus_query = {
    "query": "@(^TH$|^PP$)",
    "attribute": "acronym",
    "with_descendants": False
}
ids = region_map.find(
    thalamus_query["query"],
    thalamus_query["attribute"],
    with_descendants=thalamus_query.get("with_descendants", False),
)

then I get back an ids of {1044, 549}, exactly like I expect (1044 is PP, and 549 is TH). Note that with_descendants is False.

If I add to this query that I want to exclude PP, like the following:

thalamus_query = {
    "query": "@(?!PP)^(^TH$|^PP$)",
    "attribute": "acronym",
    "with_descendants": False
}

then I get back an ids of only {549}, which tells me that my regex has correctly excluded the PP subregion.

However, if I want to get the ids of all TH subregions, except for some, then it seems that this exclusion method doesn't work. For example, if I use the same query as above, except change with_descendants to True:

thalamus_query = {
    "query": "@(?!PP)^(^TH$|^PP$)",
    "attribute": "acronym",
    "with_descendants": True
}

then I get an ids of {1029, 262, 15, 1044, 27, 549, 300, 3009745967, 1072, 51, 1077, 1079, 571, 59, 316, 575, 1088, 496345664, 64, 321, 496345668, 325, 496345672, 1096, 1104, 599, 856, 1113, 864, 609, 1120, 617, 362, 366, 626, 629, 636, 637, 127, 138, 907, 149, 406, 155, 414, 930, 422, 170, 685, 560581551, 178, 560581555, 181, 2614168502, 560581559, 953, 186, 560581563, 444, 189, 958, 1043765183, 709, 563807435, 718, 563807439, 725, 218, 475, 733, 483, 741, 239, 1008, 1014, 1020, 255}, which DOES contain 1044, the region I previously filtered out.

I focus on the query in this case since the list of subregions is used in many different Atlas and circuit building steps, which are in the Confluence doc. Being able to use the same regex in different code places, all of which are ultimately using voxcell.RegionMap.find, is extremely convenient and allows me to keep everything consistent. Since I can't use a simpler query to exclude only certain subregions (like I tried above), I think the only alternative would be to add additional code to each package to remove certain regions. However, this would need to be done per-package, and would take time, and the current long query already solves this issue. If there IS a way to write a query such that voxcell.RegionMap.find returns a list of region ids, but excludes certain subregions even when with_descendants is True, then I can reform the query. I'm not sure that's currently possible without changing voxcell.RegionMap.find which is extremely important and I shouldn't change.

[arnaudon]

Yes, I realised the same, so I wrote a little code to get the long regex to play with:

import voxcell                                                                                          
                                                                                                        
if __name__ == "__main__":                                                                              
    region_map = voxcell.RegionMap.load_json("atlas/hierarchy.json")                                    
    region_list = region_map.find("TH", "acronym", with_descendants=True)                               
    RT = region_map.find("RT", "acronym", with_descendants=True)                                        
    print(RT)                                                                                           
    accr = []                                                                                           
    for region in region_list:                                                                          
        if region not in RT:                                                                            
            print(region, region_map.is_leaf_id(region))                                                
            if region_map.is_leaf_id(region):                                                           
                name = region_map.get(region, "acronym")                                                
                accr.append(name)                                                                       
                                                                                                        
    regex = "@" + "".join(f"^{a}$|" for a in accr)[:-1]                                                 
    print(regex)