Skip to content

System Device Tree Meeting Notes 2021

Jump to bottom Edit New page
Nathalie Chan King Choy edited this page Dec 16, 2021 · 8 revisions

Table of Contents

2021-12-08

Attended

  • Arnaud Pouliquen
  • Ed Mooring
  • Eric Finco
  • Kumar Gala
  • Mark Dapoz
  • Rob Herring
  • Tomas Evensen
  • Stefano Stabellini
  • Nathalie Chan King Choy
  • Christopher Clark
  • Ben Levinsky
  • Loic Pallardy

Agenda

  • Removal of resource groups

Notes

  • Recording link. Please download sooner than later if you need to catch up, since the recording will expire.
  • Slides presented by Stefano - because of Mailman scrubbing, you might have to copy/paste from browser into a text file and save as slides.html
  • What resource groups are & how they were used until recently
    • Have multiple domains, each with unique access to things
    • Resource group was a separate node, separate access list, for sharing things, where could be included in multiple places. Benefits: Makes it very obvious that you're sharing & avoids repetition.
  • Difficulties with resource groups
    • Access flags are bitfield describing settings for how unique resource to be assigned. These flags are domain-specific & access-specific. Became confusing very quickly & very complex to try to explain & was causing ppl to make errors all the time.
    • Hierarchical domain example with different subsystems to show how it's domain-specific. Access is done by firmware. Included a Xen domain for hypervisor w/ SMMU & IOMMU to enforce the device in linux1 domain can't touch anything in linux2 domain. In this example, the flags could have very different meanings. Sharing at different levels of hierarchies & even parent level needs access to the flags. => Where to put these bit fields for these different levels & sizes of access flags?
  • Decided to take a step back & re-design so that it's clear & simple for ppl to use.
  • What happens if we just take out resource groups?
    • We tried repeating the device for sharing in DT format. Maybe need some help from the tooling (Lopper) to ensure we don't share accidentally. We put the access flags at each domain, the flags that are meaningful to that domain.
    • In YAML, we tried to use YAML anchors
      • DT is meant to be written by few, read by many
      • Configuration (domains) could be written by multiple ppl at multiple stages
      • Anchors allows textual reference & using it in multiple places
      • Bruce: Didn't work as we had hoped. Wanted to have multiple inheritance & other things that didn't make it through YAML parser the way we had intended.
  • Alternative mechanism to achieve same benefits, without the downsides
    • Bruce: Extended YAML anchors with <<+
      • Lopper can recognize this key to do special expansion
      • Reserved memory example with expansion & key merging
      • Can automatically create numbered nodes & put properties in nodes
      • Can do multiple inheritance & have properties of main node
      • Can use 95% of the aliases & anchors & then have syntax for extended merging of dictionaries
      • Now it's part of Lopper YAML front end, so rest of the flow doesn't need to know about it
    • Tomas: Saw precedence w/ Docker & others
      • Bruce: We're not the only ones who ran into this. Docker YAML configurations are not spec-compliant & have a special parser that expands & merges later in processing. Similar, but not what we could exactly re-use from upstream.
    • Stefano: So, now (with extended YAML anchors) we can do what we were trying to do with YAML anchors. Now, can write the nodes in definitions section, then just refer to them where you need them, without causing the problems the resource group caused. Now it's just text & no semantic implication that resource group had. Now, it's like using a more flexible pre-processor.
    • Kumar: Does the <<+ operator always do the same thing & what expansion is it doing?
      • Bruce: Yes, always same. Access example shows 2 ways of doing: will take list of expanded aliases makes them into nodes. Will get expanded & get put into properties of the parent of the YAML node.
      • Tomas: Most ppl are not familiar with YAML anchors. Once you are more familiar with YAML anchors, you'll see how we thought they should work.
      • Bruce: YAML anchor & alias, like a C dereference: The properties of that node, the key-value pairs in that anchor are merged into the key-value pairs of the object where you reference that alias. "*" means just expand key-value pair. "<<" means dictionary merge. Problem: we wanted to merge multiple into one, but that wasn't allowed in basic YAML anchors.
  • Kumar: Going back to resource group DT side, flags could be specific to a given domain. How does that translate to YAML? When you were just duplicating access in each place, an issue: How to know how to interpret access?
    • Stefano: Good question that we asked ourselves. What is this flag specific for? This is what led to demise of resource groups. This flag is specific to the domain. "Access" means this domain gets access to the resources & domain can be different things so the way you define access can be different. Flags are about how you get access to these resources. e.g. Read-only might not be supported in all domains. So, flag needs to be defined with domain type. What we have now is a list of flags & bit representations, together with the definition of a new domain compatible string.
    • Note: Slide 3 has an error in linux2: domain@4 with the 0x8 flag, where there is no extra compatible. There should be something like compatible xen domain & that could be a xen domain flag.
    • Kumar: Having a property get re-defined by different compatibles: Do we have cases where property changes completely, based on which compatible you're looking at?
    • Rob: Generally we don't. There are a few cases.
    • Kumar: List of compatibles typically means this node is compatible as an openamp, domain-v1, so if my code knows how to parse & manage openamp, domain-v1, it would see access and interpret 0x1 as another device?
    • Stefano: It's an access flag cell, and if you're not compatible, you just wouldn't know how to interpret any of the numbers.
    • Kumar: How to translate access to YAML?
    • Stefano: We have some simplifications for YAML: Reg w/ address & size, dev, and flags. Flags could be number, or have read-only: true. Lopper can implement & knows how to interpret.
    • Note: Slide 4: Compatible for linux2 has a mistake
  • Loic: You are mixing different means to protect the peripherals: SMMU configurations with flags & peripheral firewalls. Confusing. For me, the access is for the firewall. SMMU is quite different w/ Linux programming SMMU according to allocation of buffer & how IP would be able to access the rest of the memory.
    • Stefano: IOMMU may be a bad example b/c brings another dimension of problem b/c dynamic configuration. Let's talk about System MMU only. If we have 2 different firmware, we'll have 2 different set of flags.
    • Loic: So you want to have in the same location (e.g. domain) the definitions that the ethernet can be accessed by domain in __ but another IP could be shared between different processors. Then system firewall be able to manage level 2 of the System MMU of your processors limiting the access rights of your VM, for example.
    • Stefano: Yes. e.g. There are some flags we have in firmware that are firmware configuration, e.g. time-share flag will allow resource to be shared in time between different sub-systems. This is specific to this firmware, which only makes sense in a Xilinx subsystem domain. In Xilinx customer's firmware, will still have System MPU, they will have System MPU flags, but not time-share. So, the flags are specific to the domain.
    • Stefano: IOMMU is difficult conversation b/c it's programmed dynamically. It's in the example though to show that it's got different protection mechanism. Another mechanism for protection (difficult but possible) is bus remapping. Point is that we have different types of accessibility & therefore different flags.
    • Loic: What's is missing is how all this would be processed to create the different configuration tables, e.g. for the firewall, to configure the VM, System MPU, System MMU, etc. For each peripheral, would have to pass all the domains to get the flag & to build for each peripheral if it's present or not & to build the configurations for HW protection.
    • Stefano: Correct. We are currently working on it. Can discuss next time & show example.
    • Tomas: Mental model for these flags: It's really info a specific client. In Lopper, you have a back-end for Xen, for example, that interprets these flags & spits out config file. Those flags would not show up in a pure DT later on. More to have all the info in 1 place in System DT.
    • Loic: We need to have some rules, maybe in Lopper, for sharing IP that are secure/non-secure, to help customer verify the configuration.
    • Stefano: We do have a secure flag.
  • Let's have next call in Jan/Feb & can discuss secure.

2021-09-13

Attended:

  • Arnaud Pouliquen
  • Bruce Ashfield
  • Etsam Anjum
  • Joakim Bech
  • Kumar Gala
  • Mark Dapoz
  • Marti Bolivar
  • Rob Herring
  • Tanmay Shah
  • Tomas Evensen
  • Stefano Stabellini
  • Nathalie Chan King Choy
  • Ed Mooring
  • Sergei K

Agenda

  • Secure & non-secure address mapping differences for same node
  • YAML

Notes:

  • Meeting recording (please download it sooner than later if you need to catch up)
  • Slides: (to get from Stefano)
  • Secure & non-secure address mapping differences for same physical peripheral or DT node
    • Stefano: Don’t have anything that works today and no trivial way to solve it
    • Rob: Peripheral looks same but shows up at different addresses?
      • Stefano: Yes
    • Stefano recapped what we have today in System DT, so we can understand what we have to build on top of
      • Address map & CPU cluster
        • Meant to solve a different problem: have multiple CPU clusters in heterogeneous system, some of which might have different address maps
        • Address-map: Works similar to ranges for buses, but translates from parent address space to CPU cluster address space. Mostly useful for CPU-cluster-specific different mappings.
      • Domains
        • Today, don't use it to express address mapping differences at all.
        • Put together CPU, memory, devices -> unit of HW resources that are used by a SW stack
        • Specifying EL: Only way to specify in System DT today to specify secure vs non-secure
        • But, Domains are not for HW description, just config. No address mapping to indicate it shows up at a different base address.
    • Example: GEM
      • How to describe different start addresses for GEM in normal world & secure world?
      • We have no way to describe it. Address map has no secure/non-secure distinction, just a per-cluster distinction. EL info in domains, but no mappings inside domains. Neither will cover it.
      • If we had to use address map, some possibilities:
        • Could extend it for normal/secure world mapping
        • Could have secure-address-map in addition to normal address map
      • Zephyr didn't like it b/c address-map is for address mapping differences for each CPU cluster. What we're trying to solve is between secure & non-secure and would be same across all CPU clusters.
      • Their idea, polished by Stefano, to seed the discussion:
        • Why don't we express differences in base addressing directly as under the device node.
        • In addition to reg, could have secure reg property.
        • Then could specify different base addr for secure world.
        • Looks a lot simpler than huge list under address-map
        • But, ends up bigger than Stefano thought. Has big impact across the DT. Even if we have a secure-reg property, we would still need a secure-address-map. What if there are differences in mapping between secure & non-secure world between CPU clusters.
        • Secure reg: still need secure ranges to apply translations to secure addresses
    • Rob: Other proposal of using ranges under simple bus?
      • Stefano: We don't need secure reg if we have secure ranges. Could specify at bus level instead of for each device. If instead we have secure reg, still need secure ranges anyway. Can't get away without secure-address-map.
    • Rob: Suggest to define new bus type simple-secure-bus, with 1 more address cell for parent bus & define secure/non-secure in that top address cell.
      • Stefano: Just reg alone
      • Rob: Just ranges too, probably. OS would have to know to look for ranges entries with top cell set to 1 for secure, 0 for NS. It's not unheard of. ISA bus defines 2 cells. PCI has 3 cells.
      • Stefano: Solves ranges vs secure-ranges & reg vs secure-reg. Both can be united into single property.
      • Kumar: Agree w/ that route. Works great for addresses. If have to remap interrupts based on secure vs not, then how to do that?
      • Rob: Interrupt map can take an address, but not used nowadays. Think would take child address unless you did it another level up.
      • Kumar: Does interrupt number change from secure to non?
      • Marti: Not sure.
      • Rob: Generally, you could have any property different for secure vs. non, e.g. could set up power domain for just secure.
      • Stefano: If the nodes describing the resource are completely different, one could even duplicate them. One instance under secure bus, another instance under simple bus.
      • Kumar: HW is still 1 physical resource. Don't want to accidentally indicate 2 different Ethernet controllers when only 1 physical resource.
      • Rob: Different view of HW for secure vs. non
      • Stefano: Today we wouldn't know they are 2 faces of same HW resource. Would need a way to link them together to indicate same HW.
      • Ed: Are there other things besides secure/non that cause different views of the world?
      • Tomas: Could different EL's see things from different address?
      • Rob: Don't think that gets exposed out to the bus. Older: User vs. supervisor mode would get exposed. There are CPU clusters having different views of peripherals.
      • Stefano: Bus txn have few bits to pass extra info, but EL isn't. Caching attributes are. One could theoretically expose a different view via different HW access path based on caching attributes.
      • Mark?: Intel has system management mode. So there are other modes to consider.
      • Rob: Would have to consider if have to share the DT across those modes.
      • Stefano:
        • Would be good if System DT could. But, can't solve easily today.
        • Have 1 Address map per CPU cluster. If were to extend address map w/ 1 more cell, would have to pass an execution mode parameter to abstract away from just secure/non. On x86, could include system management mode. You could define other modes on other architectures.
      • Rob: Same for using ranges & adding address cell there. Have 32 bits of info you could encode.
      • Kumar: Instead of secure-simple-bus could be executionmode-simple-bus with this extra field w/ bits. 1st bit we define is secure/non. Management vs not could be another bit.
      • Stefano: Unless there are significant differences for each CPU cluster, you could prob get away with 1 address map row that covers biggest range. Instead of detailed list at address map that's duplicated, could have secure-bus, then address map will be simpler in the simplest case.
      • Kumar: Could say extended-simple-bus
      • Stefano: Lopper could parse & generate simple bus output DT for regular OS, so they don't need to know about different execution modes.
      • Kumar: Yes, OP-TEE or TZ might want all that info, but Linux might only care about simple bus. Good for Lopper to give the option.
      • Joakim:
        • OP-TEE doing some runtime modifications to DT. Add reserved memory node for what's reserved in secure mode, if it's not there already, and pass it to Linux kernel.
        • Carving out ranges of memory for secure & defining what Linux can see.
      • Stefano:
        • That's do-able with System DT using domains. Can have domain for OP-TEE, Trusted firmware, bootloader & Linux domain. Under domain, can specify what range is for OP-TEE, TF-A, Linux. Works b/c just ranges & configurations.
        • Took for granted that these addresses are not configurable. These are HW properties. So, makes sense to do it outside of domains.
  • YAML for DT
    • Presented at Linaro Connect LVC21F
    • Does Rob agree w/ direction so far?
    • How to define new simplifications?
    • Had to write it as a spec => Realized there are 2 aspects
      • Translating YAML to (system) DT
      • Simplification to make YAML simpler, e.g. specific to reg or other properties
    • DT nodes are YAML key-value pairs
      • Content of node is value
      • Key is label when available, else is node name
      • Properties are key value pairs
      • Hierarchy maintained by indentation
    • 5 simplifications
      • Could add more, but then they'd become binding-specific
      • No quotes for strings
      • Reg: Explicit start & size for each range
      • _cells not useful w/ YAML b/c YAML can have sequences
        • Kumar: How does that work if you have 32-bit # for address cell, but your address cell for system is 64-bit?
        • Rob: That doesn't matter. Can have 1 cell on 64-bit processor. Only exception is for DMA mask calculations.
        • Rob: For most cases, could do this with existing DTS. Example is 4x3 matrix. Can do < > and have equivalent thing. For address, can calculate max address size I need for a given bus.
        • Kumar: Should the YAML be binding aware? I think it should be.
        • Stefano: More example at end.
      • Don't use phandles, just use references
    • Binding aware example: Break out the encoding, which requires binding-specific awareness.
      • If each binding could add a language to specify its own simplifications
      • Then can have Lopper translate YAML to DT and then it will work
      • Kumar: Can name cells & that's generic. Flags.level, flags.trigger, flags.mask? 1st level is naming the cells
      • Rob: Binding schema today have # interrupts & what they are. What's in each entry is opaque b/c intc-specific. We have no way to validate that.
      • Stefano: Picked interrupt as example b/c no cross-controller or generic way to define the interrupts. If we want to make further simplifications, we will have to have intc-specific simplifications & add info to gic binding on how to do the translation.
      • Rob: Interrupt cells property & interrupt controller schema would define a schema for the cells. Tooling would be aware of how to parse properties & look up intc & find schema & apply it.
      • Kumar:
        • In Zephyr bindings, we have named cells.
        • Could do type, number, flags. Don't have 2nd level - fielded type is not supported yet, but could be a way to add 2nd level. Has been shown to be pretty generic.
        • Any property w/ cell-based thing (clocks, pintctrl, etc.) could have cells named in binding, to allow tool to provide it back out to user for checking/code gen/YAML. Once we define it in schema, and add 2nd level, you could do all this in the tooling generically.
        • Today, Zephyr has its own binding language. Would like to eventually converge to use same schema language & bindings used by Linux.
        • Difference today between Linux binding & Zephyr binding: Zephyr binding has concept of naming fields of a cell structure.
      • Rob: In Linux, today, it's free form text in the interrupt controller, interrupt cell description.
      • Stefano:
        • That's our starting point.
        • In some bindings, could come up w/ way to replace free-form English to use YAML. Could then make tool that translates.
      • Kumar: Zephyr binding:
        • Line 17 onward: Could do something similar. Tools, YAML could use these names. Cell # is implied by how many fields in binding.
      • Stefano: Once we add 2 levels, that would cover it.
      • Kumar: Working on something in schema format language that Rob's got to add a standard way of describing cells. Maybe also way to describe a bit fielded value. Having those 2 things could allow YAML tooling to use the schema info to validate.
      • Stefano: Makes sense to me.
      • Rob: Not sure if breaking out the fields actually simplifies.
      • Kumar: Benefits: Allows the code generation to be consistent. Tooling in YAML case - if you just want to change interrupt priority, today in DTS have to re-write the whole field. YAML would be convenient way to just set interrupt priority of 6 for this device, without having to specify all the other values again.
      • Stefano: One thing for a DT expert to change priority. Disaster if non-DT expert goes in to change priority ("drive-thru" modification). Harder to make mistake w/ YAML w/ explicit fields.

2021-07-22

Attended

  • Tanmay Shah
  • Loic Pallardy
  • Mark Dapoz
  • Stefano Stabellini
  • Nathalie Chan King Choy
  • Christopher Clark
  • Ed Mooring
  • Etsam Anjum
  • Kumar Gala
  • Marti Bolivar
  • Bruce Ashfield
  • Bill Mills
  • Tomas Evensen

Agenda

Action items

  • Stefano to share the slides from today
  • Stefano: Look at DTC to see how to deal with node having multiple labels.
  • Stefano: How to specify more data driven way for each binding maybe using YAML schema to specify this kind of language to end up with interrupt #, interrupt type, etc. Consider doing templated node.
  • Nathalie: Schedule next call for shortly after Linaro Connect

Notes

  • Link to recording. If you need to catch up, please download for viewing sooner than later, before the recording expires.
  • Recap
    • Nested domains
      • e.g. hypervisor & guest
      • Domain class example of nested domain: HW based domain is available on Xilinx boards (subsystems)
    • Editing DT is difficult for folks new to DT
      • Simplified YAML
      • Discussion w/ Kumar & Rob: We want a more formal definition of the YAML and can we make it more generic to cover more than domains? Have taken a stab at that, so most is generic & only a bit is domain-specific. Functionally equivalent to DTS.
  • How is it defined?
    • DT nodes are mappings in YAML key-value pairs
    • Hierarchy is preserved using YAML indentation
    • "mapping" in YAML has a specific meaning
    • Node name vs. label
      • DT source has both
      • YAML can't easily have both. Stefano prefers label b/c more clear what it is compared to node name. Also, node name can usually be programmatically generated.
    • Simplifications
      • Strings: Don't need quotes in YAML
      • Reg and address ranges: Can just have array of address starts & sizes as key-value pairs. Can also use human readable formats (e.g. 2M, 4G, 1T) for both start & size.
      • *_cells: Don’t need #address_cells and #size_cells in simplified YAML and can just generate them. e.g. interrupts as sequence of sequences, so can get rid of #*_cells in simplified YAML.
      • Phandles: not used in simplified YAML, only references (easier for humans to read/write). & is not used.
      • Booleans: Using true/false.
  • Full example: about a third of ZCU102 DT done by hand by Stefano
    • Simpler to read
    • More intuitive to write, without knowing 32-bit cells rule, splitting, etc.
    • Bill: When you generate DTB, you still need to know if you need 64-bit or 32-bit address. That info has to come from somewhere.
    • Stefano: Purpose is to convert to DTS or DTB and should not
      • Don't think it's needed b/c DT is always 32-bit cells. So, the conversion tool can calculate tha ddress & size cells and put in the right place, as long is it's consistent.
    • Bill: Is it just conveying the value, or conveying the type. Cells_1 vs cells_2 is conveying type info, which could be expressed a different way.
    • Stefano: In regs, was positional, now is explicit. In interrupts, can make further simplification w/ interrupt # or type columns, like regs.
    • Bill: Wondering if we should have reg32/reg64 or start32/start64
    • Kumar: What's the intent? Get the domain simplification & if can represent everything in YAML.
      • Who's going to write something this way?
      • Who's specifying what?
      • You have info that would come from bindings. Some of the info may be static.
      • How far to go wrt what's user-configurable data (e.g. not interrupt number, but can configure interrupt priority)
    • Stefano: Simpler language. We are not the target audience b/c this group is experts. Point of simpler language is domains, which is definitely config which is done by role who rarely uses DT.
    • Kumar: Get desire for simpler human readable/writable language but who's reading/writing? Many generate DT from design. SoC is less of a concern. Interrupts is still not that readable - think would need names (e.g. num, type). If you got that info from bindings - maybe binding could give you more template info to know what's user-configurable.
    • Mark: Other advantage: Almost all modern editors (e.g. Visual Studio) can understand YAML natively in the IDE. Could have schema behind that would enforce. Think moving to YAML is really good. Much easier to plug into the IDEs.
    • Stefano: Library to parse language.
    • Stefano: One of the biggest benefit: To specify flags & bitmasks with names. Can be extensible.
      • Most boards on Arm have GIC, so could have interrupt type & number or let it be open & let ppl add info to the bindings or simplification to the tool for proper translation
    • Kumar: Think the higher level language should assume the bindings are there, so you don't have to hard-code how to handle start & size. Should be data-driven, so any cell-based list can have cell names specified.
    • Stefano: Some things in epaper original spec, but others not (e.g. clocks depend on clock controller). Would be good to have data-driven way to add things like clocks, interrupts.
    • Tomas: Agree on data-driven.
    • Bruce: We've been searching for the right way to describe in Lopper. Have hacked up prototypes, but haven't found the right way yet to not make the descriptions more complicated than the YAML. Have a framework, but no solution. Currently, little python snippets do it.
    • Bill: Interrupt example - is the only way to interpret the interrupt sequences to look at the interrupt parent?
    • Stefano: Yes, that's how it is on DT.
    • Bill: You can do what you've done here without understanding hte data itself.
  • Etsam: Are there any access attributes with these fields? Virtualization use case: Can change for virtual device, but not for physical device? How to validate you didn't make a bad change?
    • Stefano: Have not tried to describe virtual devices yet. It is an interesting problem that is even in DT, rather than a YAML problem. Have discussed validation w/ Bruce.
    • Bruce: Lopper does a little bit of validation on the YAML.
    • Etsam: Agree, same problem in vanilla DTS. Think we'll need some kind of schema to validate.
    • Stefano: It is a goal for the tool to check the validity. Started discussion for domains. Person editing domain is likely not to be a DT expert, so prone to errors.
  • Kumar: YAML syntax: Can you deal w/ fact that node can have multiple labels?
    • Stefano:
      • Maybe not unless we introduce something special.
      • It never came up during domains use case
      • Making more generixc to rest of DT only started in last few weeks. Right now, have to choose between node name or label.
      • If it is important, can try to think of way to do it.
    • Kumar: Can look at DTC code to see what it's doing to output YAML
    • Stefano: Look at DTC to see how to deal with node having multiple labels.
  • Bruce: Lopper update
    • Domain: Can take domain YAML and expand it fully into DT format with some Xilinx assists doing some of the general work. Generate labels, reorder nodes, expand nodes, expand flags.
    • Once we start doing tree manipulations, it's on DTS. YAML is just an input format.
    • May need another stage in pipeline to do more expansion before being able to get the internal DTS for manipulation.
  • Stefano: How to specify more data driven way for each binding maybe using YAML schema to specify this kind of language to end up with interrupt #, interrupt type, etc. Consider doing templated node.
    • Kumar: Interesting precursor: Pull today's YAML schema for Linux and have a templated node (whether YAML, DTS) to see what can be filled in. Could be interesting starting point as some kind of Python code. Then not too bad to expand the schema to add some preperty info for things like cell names.
  • Stefano: We made a presentation submission to Linaro Connect LVC21F talk on this, so may have a wider audience in that forum & would probably have more complete Lopper implementation & example to show.
  • When to host next call?
    • Linaro Connect is early September 8-10
    • Stefano is away 2 weeks in August
    • Nathalie: Schedule next call for shortly after Linaro Connect

2021-06-22

Attended

  • Stefano Stabellini
  • Bruce Ashfield
  • Christopher Clark
  • Ed Mooring
  • Etsam Anjum
  • Rob Herring
  • Sk
  • Tomas Evensen
  • Nathalie Chan King Choy
  • Arnaud Pouliquen
  • Loic Pallardy
  • Kumar Gala
  • Dan Driscoll
  • Krzysztof Kepa

Agenda

  • Quick summary of last time
  • YAML representation for subset of the Device Tree

Action items

  • Stefano to share the slides
  • Stefano: Try to write a schema for the simplifications as 1st step in generalization
  • Nathalie: Schedule next call for end of July

Notes

  • Recording link (please download sooner than later, to avoid disappointment when the Webex recording expires)
  • Summary of last time: Domain hierarchy
  • YAML
    • DT: Meant to be edited by humans, but is hard to do by hand & get it to compile
    • Different from the full YAML representation of device tree. Goal is quite different: Customer demand to make it easier for humans to read/write, while also keeping it easy for tools to parse.
      • Only for domains b/c that's the part that gets edited & in multiple steps
      • The rest of DT is almost static
    • YAML is more intuitive by DT and already in use by DT community
    • Lopper can take YAML as input for domains part
    • Made simplifications to make it easier to read/write, so not 1:1 of System DT representation
      • Bitfields: Simplified for humans by naming the flags
    • Showed example of xen domain
    • Made domain keyword explicit (note: typo in the example, accidentally duplicated "domains" keyword) to help separate sub-domains from other configurations
    • Rob: Schema bindings in YAML -> Schema bindings & JSON schema. Within JSON schema ppl use JSON files or YAML. We went with YAML.
    • Stefano: Zephyr schema different?
      • Rob:
        • Zephyr schema is different.
        • DTC YAML is purely an intermediate format for schema checking & subject to change. DT encoded in YAML.
        • Everything is array or array of arrays b/c DTC doesn't know anything about the type. Just arrays of bytes, or integers, or strings.
    • Stefano: These YAML have very different purposes
    • Bruce:
      • Lopper internal tree structure stood up based on this. Eventually gets merged with the other components that come in as DTS file. Then can output things back into the different formats.
      • Lopper expansions: Brings it back to 1:1, as if it came in via DTS.
        • Stefano: i.e. some pre-processing to bring it to 1:1
        • When you dump it out, the simplifications are lost
    • Rob: Concept is interesting. Can we generalize it for all DTs? To quote Olaf: "DTC is an assembler not a compiler". Not a high level language. So, if we came up w/ a higher level language for specifying a language to specify DTs in, then compiled to DTBs. One way that could work is using the schema.
      • Working on using the schema to get the type info back if you only have a DTB.
      • Arm System Ready IR for testing platforms & having standardized firmware on IOT Linux platforms
        • How would you test the DTB that comes with the firmware? Can't do that with the schema yet b/c relies on source info.
      • Stefano: So can only test from DTS, not DTB.
      • Looking at using the schema to add the type info back, then run the validation w/in the constraints of the schema.
      • Could do something similar here: Look at schema to know type of property, then output the right size for it.
      • Stefano: Could use info from schema to fill in things like address cells. In this case, Lopper is doing it, but in that case can come from the schema.
      • Stefano: If you drop the parts that are domain-specific, the rest of the simplified YAML could apply to a generic DT
      • Bruce: Explicit flag names: Would like to have a way to map names to fields in a good way w/o it knowing about them. Could generalize to go back & forth
      • Kumar: Would want to encode in DT schema
      • Bruce: Yes, should use same schema mechanism
      • Kumar: Don't think there's a way to describe field this way
      • Rob: Could use YAML tags. That's what YAML encoded DT output uses, primarily for type sizes (8, 16, 32, 64-bit) and phandles. (!phandle in front)
      • Kumar: DTS YAML would look like that, but what would schema YAML/JSON look like
      • Rob: Take tag & save size & compare w/ schema keyword
    • Stefano: One question that came up often: labels vs domain names. We ended up w/ extra domains key in YAML b/c in DT have labels & domain names. One of the 2 has to be dropped here b/c we just have the key, which is more like label. Do you have an opinion?
      • Rob: YAML references could probably map to that. So far, have avoided that issue.
      • Compatible is there, which is most important. Simplest thing to do is to use the name, but it's the least informative thing & thinking about what the human wants to write => use label as key b/c carries the most info (e.g. linux1, not
      • Rob: Could just use the labels & get rid of domain except top level domain. The subdomains have compatible. Tooling could know that compatible should have node name of domain. Then transform the Xen label to node name based on compatible, when output.
      • Bruce: Lopper similar: keeps domains, then flips name to label & send it out.
    • Bruce: True/false. False is absence of a property in DT, so would drop into the DT, but then lost info on round trip. Have a flag to maintain false values. What's ppl's opinion?
      • Rob: False means property is absent in DTS
      • Kumar: Can you have pedantic mode w/ all the falses in there?
      • Bruce: Yes, have the flag. Was not sure what the default should be.
      • Rob: If you have a high level language & compile it into something else, you expect you lose stuff & can't get it back on round trip. So, maybe it should be a non-goal.
      • Bruce: Default is to throw it away & have flag if you want to maintain it.
  • Bruce: Lopper update & YAML
    • More YAML support for domains -> intermediate representation
    • Everything is on lopper repo https://github.com/devicetree-org/lopper
    • Zephyr dtlib can now be used as parser w/o needing dtc to read in dts files
    • Making lopper easier to pull in & use: pypi installation
    • Putting more files in for reference pipeline for System DT specification
      • Want to do more on code generation
  • Stefano: I'm maintaining a list of differences between YAML & plain DT
    • More detailed than what's in slide 5
    • Majority are generic DT, then few more for domains part
  • Any suggestions for next steps?
    • Rob: If it can be generalized more
    • Kumar: Is there any manipulation that we can do, like what DTC can do (add/delete node, change property), or does it make sense to do that some other way?
      • Stefano: Have been just using Lopper so far. Have not looked at overlay or runtime change.
      • Kumar: There's not good way today to change the name of a node with DTC.
      • Rob: For simple cases, could generate node names & addresses if you have schema w/ class of device or node name pattern. But then what to do with all the ones that don't fit that?
      • Stefano: Simplified YAML helps b/c puts less emphasis on node name. Node name is almost always generated from compatible, but we put label b/c more interesting. Haven't tried using simplified YAML for full DT, just domains part.
      • Kumar: Maybe need extension to schema to figure out how to specify a class of devices (e.g. these are all SPI controllers) so the tooling can map from schema to spi@address.
    • Stefano: To make simplified YAML more generic: Add to schema extra meta info for Lopper to do the expansions. What would need to be moved to schema from lopper?
      • Bruce: Don't have complete list: mainly the flags, different memory modes might apply
      • Stefano: Size w/ no address for memory (telling Lopper to figure out the address for me b/c don't care)
        • Rob: We have for reserved memory (separate property)
        • Rob: Where does Lopper get the range of addresses to use?
        • Stefano: Higher up in the domains
        • Kumar: May need some tags for constraints (alignment, allocate from top/bottom)
        • Bruce: Things we can do there for sure
        • Kumar: In 1st pass, just specify start & size for now & can come back on 2nd pass to leave off address
        • Stefano: Agree, can leave more complex problems like this one out of the 1st pass
  • Kumar: Getting type info out for DTB: Is there a python data model view to see all the info for 1 compatible, to know all the schema properties we expect?
    • Rob: Believe compatible is based on how we find the type info. Search through schema & match & come up w/ dictionary that says this property is this type.
    • Kumar: Could you say for a compatible that a particular key isn't here from the schema & add it back? Issue w/ Zephyr bindings was that JSON was more geared toward validation, and not for code generation.
    • Rob: There are some tools for JSON schema to generate data out of the schema & generate schema from data, but haven't …
    • Kumar: How to check that you're not setting a field that doesn't exist?
    • Rob: You'll need a schema for your high level.
    • Stefano: Happy to write a schema, but problem is that it's more of a list of simplifications. Would be a schema of the simplifications & rest is duplicate of the schema.
    • Rob: Part of the complication in our schema is b/c everything is an array, but the tooling adds that in. Also, you're referencing another schema w/ the complications.
    • Stefano: Try to write a schema for the simplifications as 1st step in generalization
  • Nathalie: When to have next call, factoring in vacation period?
    • Looks like end of July should be good. August will be vacations.

2021-05-10

Attended

  • Bruce Ashfield
  • Christopher Clark
  • Daniel P Smith
  • Stefano Stabellini
  • Nathalie Chan King Choy
  • Krzysztof Kepa GE Research
  • Loic Pallardy ST
  • Mark Dapoz
  • Ed Mooring
  • Tomas Evensen
  • Joakim Bech Linaro
  • Maarten Koning
  • Grant Likely Arm

Agenda

  • Christopher and Daniel will talk about their Device Tree bindings for HyperLaunch, which has very close similarities to System Device Tree domains.
  • We'll also discuss the topic of hierarchical domains
  • Bruce will present the latest Lopper updates.

Action items

  • Stefano: Send slides
  • Bruce: Send out the Lopper YAML grammar

Notes

  • Recording: (I am not sure how long before the recordings expire or I will hit my storage limit, so if you need to catch up by watching the recording, please download it in the next couple weeks) Recording link System DT call
  • Slides: Stefano sent out to the mailing list
  • Hyperlaunch
    • Stefano: Intro
      • System DT and Xen Dom0less
      • This meeting will focus more on execution domains
      • System DT domains were inspired on older spec for Xen Dom0less, which was DT-based spec for Xen for describing Xen VMs to start at boot time. Similar concepts & very much aligned.
      • In next few months, would like to work more on hypervisor side
      • Daniel & Christopher worked on Hyperlaunch (Xen Dom0less spec)
    • Christopher, Daniel
      • Started with several years of OpenXT work
      • Building for both X86 & Arm systems
      • Dom0less does some dynamic generation of DT for Arm guests for VM config
      • Flow we've seen in System DT doc w/ Lopper suggests static system config
      • Hyperlaunch looking at enabling bootloader to do some dynamic generation of the DT that gets passed to Xen to describe VMs, so can do more at runtime.
    • Stefano: To describe domains on multiple execution levels, incl. trusted firmware domain on Arm
    • Christopher: Moving away from coarse grain privileged Dom0 & non-privileged DomU. Will send finer grained description, based on DT: e.g. domain to
      • ___.
      • Deprivileged guest VMs
      • Isolation VMs to protect other VMs on the system from adversarial activity from that network device.
      • Stub domains: Device emulation in isolated environment.
    • Christopher: One use case in mind: Virtual TPM enables isolation between VMs that are holding secrets on behalf of guest. No all-powerful Dom0 that has access to HW & all the confidential info. Requires sequenced launch to provide that env.
    • Showed example:
      • Older version, bit more x86 specific, but newer version will come out that is more generic
      • Hypervisor node would go under chosen
        • Would allow us to crop this node if we wanted to remove this info from DT
        • Defined some containers
    • Hyperlaunch would like to be able to use SystemDT if possible
    • Stefano: Think we could align them well enough so there would be just 1 spec
  • Hierarchical domains
    • Higher privilege domain on top, lower privilege below
    • Xen is an obvious example of more privileged domain (hypervisor at EL2) with children (EL1)
    • Beyond Xen, other examples:
    • Xilinx firmware has concept of subsystems
      • Collection of HW resources that power up together & have same lifecycle -> belong to same "domain", which are so close to system DT domains, but instead of hypervisor-defined they are firmware-defined
      • Xilinx subsystem can include cortex-R or cortex-A in 1 subsystem
      • Looks like clean example as top level domain of hierarchy
    • If other vendors have similar firmware-defined domains, System DT can work
    • Subsystem: ID is really necessary. Want to add to other domains.
    • ATF is running at higher privilege than Linux, so makes sense as parent of Linux
      • Tomas: By expressing it this way, you inherit what's in the inner loop, so you don't have to repeat b/c ATF has access to what's in the children. Helps you see shared devices & memory.
      • Daniel: Interesting w/ nested virtualization on Arm platform
      • Stefano: Challenge we haven't solved yet for virtualization: How to distinguish physical devices from virtual devices. Will take some iteration.
    • Resource group: shared between multiple domains (e.g. SRAM)
    • What about memory that is not main memory?
      • There are a number of memory types on SoC that are not DDR
      • e.g. MMIO SRAM: how to specify these notes in System DT under domain?
      • Adding in access list like other device: But it's memory, so may want to carve up into subsets - need address & range.
      • What if we consider it as normal memory? "Memory" property is really meant for main memory
      • Took inspiration from openamp remoteproc binding: Uses property called "sram"
    • Joakim: Like tree idea. But subnodes on exception level?
      • Stefano:
        • Higher privileged domain is parent & lower is child. Typically tied to EL. ATF is secure EL3, so is parent & Linux A72 is normal level EL1.
        • In scenario with no EL, top level specify as secure EL3, but it's not really informative in this case
    • Joakim: If you have list of different memory regions, it can be helpful to see who has been taking which memory are. Hard to see in Linux what memory you're not allowed to touch.
      • Stefano: One reason why can be helpful. Can describe what's present but not meant for the OS.
      • Tomas: Lopper will warn if you use the same memory in different domains. But, if it's child using the memory, it's OK unless you say otherwise. Static config w/ Lopper can give you these checks.
    • Stefano: Memory & access works bit differently with hierarchy: You keep carving out for 1 domain or another - top of set at top & then further down is smaller set. Device access list: Uniquely assigned.
    • Mark: Wouldn't you want to do that we domains & device access list? Can get checks.
      • Stefano: Yes, we thought so too. But it became cumbersome & didn't give a lot of benefit & made it harder to read.
    • Stefano: Virtual CPU is still an open question
  • Bruce: Lopper update
    • WIP, adding more utility & functionality
    • Can take
      • DTS, DTB
      • YAML representation of the domains: Easier for tools & humans to edits. Have defaults & assumed values.
    • More stages in the pipeline, which allows us to do different things on the tree
    • Working w/ Zephyr tam to bring in their dtlib/edtlib processing for pure Python, so won't need dtc, cpp
    • Making Lopper on pypi to make it easier to install once the Zephyr work is done
    • Will have a reference pipeline in Lopper repo to show
    • Example slide: Custom back-end. In lopper repo now.
    • Mark: Do you have a spec for the YAML grammar?
      • Stefano: Yes. Knew we'll run out of time
    • Grant: Are you suing same YAML schema as rob?
      • Stefano:
        • Looking at the schema & starting to write the spec to convert to more & more schema
        • Lopper YAML is not meant to describe the spec: More a 1:1 correspondence to DT
      • Grant: There is a YAML representation of DT that's being used for the schema checker. Can the 2 representations be unified.
  • Next time:
    • Stefano & Bruce present YAML

2021-01-29

Attended:

  • Tomas - Xilinx
  • Stefano - Xilinx
  • Frank Rowand - Sony
  • Dan Milea - Wind River
  • Dan Driscoll - MGC
  • Krzysztof Kepa - GE Research
  • Bruce Ashfield - Xilinx
  • Kumar G - Linaro
  • Loic Pallardy - ST
  • Ben Levinsky - Xilinx
  • Hannes Tschofenig - Arm
  • Rob Herring - Arm
  • Mark Dapoz - Wind River
  • Ed Mooring - Xilinx
  • Arnaud Pouliquen - ST

Agenda:

  • I would also like to cover again the remoteproc bindings and they get generated from system device tree.
  • I would like to continue the discussion on the bus firewall bindings. I sent an email to the list with the summary of the past discussions.
  • If we have time, we can discuss how system device tree aligns with ffa partitions definitions.

Action items:

  • Loic: Check HW lock bindings
  • Stefano: Will try to combine the 2 properties into 1 & come up with some examples.

Notes:

  • Recording: (I am not sure how long before the recordings expire or I will hit my storage limit, so if you need to catch up by watching the recording, please download it in the next couple weeks) Recording link System DT call
  • Slides: Stefano sent out to the mailing list
  • Announcement: Linaro Connect session accepted: DTE state of the union from 4 perspectives: Bill Mills, Kumar, Stefano, Wind River
  • Stefano: Remoteproc bindings have made good progress in the past year
    • Showed original Xilinx remoteproc binding & Ben recently posted to upstream revised Xilinx binding based on feedback (thanks to the reviewers)
    • Showed TI K3 binding & is much more aligned. Stefano has slide listing the commonalities.
    • Kumar: What is Linux doing with sram property? Can be helpful to specify the property outside of Linux.
      • Ben: Application specific, or IPC
    • Rob: R5 is common from Arm & have 2 different bindings for it between Xilinx & TI
    • Loic: Agree we can find common foundation between the drivers for the co-processors
    • Rob: There are common parts. Focused on R5
    • Kumar:
      • There's value in defining a common binding.
      • Remote proc state machine could be common too
    • Rob: There is a start on work on that. Have acked.
    • Showed System DT representation for Xilinx bindings. Interesting part is in domains node.
      • Access: Give unique access to something
      • Include: Include shared resources
      • Pass to Lopper as input. Ben created Xilinx-specific plug-in in Lopper repo, assists.
  • Stefano: Bus firewall bindings
    • Another way to call it is is system-wide MPU
    • Doesn't fit interconnect bindings well
    • But maybe very similar to IOMMU bindings
      • What's missing: How to specify the slave side connection (e.g. firewall property)
      • Currently assume IOMMU covers everything in address space
    • Rob: Another issue: What if you have an IOMMU & a firewall? Probably makes sense for these both to be separate.
    • Rob: Why can't bus master ID & firewall be same property?
      • Stefano: Could, if we had a way to distinguish them
      • Rob: That node defines what each one is.
      • Stefano: Could define positioning
      • Loic: Driver in Linux must be able to check what it's allowed to access IP register before probing.
      • Loic: Need to investigate if can do at kernel level, if we can't add this new property.
      • Loic: If have several drives, have to know know which reg value
      • Stefano: Makes sense to have slave side info there, to know what's protected. Shouldn't have to encode in drivers or Lopper - can. Xilinx lopper plug in to pre-generate configuration of bus firewall @ low-level details. Could be hard-coded in Lopper plug-in to cover slave side, but makes System-DT description incomplete.
      • Kumar: Makes sense to describe b/c non-Linux SW cases. TEE or boot firmware might be configuring the firewall, and needs the info.
    • Kumar: What is firewall property in CAN case?
      • Stefano: MMIO region starting at 0xff06000 is protected by system-wide MPU. So, if you want to block everyone except 1 master to make DMA txn for that region, you have to go to lpd_xppu. Unlike w/ IOMMU, not all areas are protected.
      • Loic: Need additional ID for ST b/c not based on memory region, but rather peripheral identifier. Have hard-coded entries in peripheral firewall, similar to clock & reset. -> firewall-cells after phandle in firewall property. Other SOC vendors have HW semaphores to get access to peripherals, so SW that wants to access peripherals should 1st do something at firewall level to get access & then do something.
      • Rob: HW lock binding exists for that, but don't have to use here.
      • Loic: Will check HW lock binding. Think it's complement. Firewall adds check of access right + lock.
    • Kumar: 2 ways in DT to describe directionality. 1 way: memory & DMA ranges for in & out view that's CPU-centric. Interrupts: Interrupt nexuses w/ graph-based view, where we don't separate direction based on property name. Should we move more to interrupt way of describing things?
      • Rob: Should be able to combine into 1 property & say there's always at least 1 cell & that cell says master or slave & subsequent cells are ID values.
      • Stefano: that would make it easier
      • Rob: Not sure if you need master & slave ID. Could walk the list to find firewall properties & firewall property entries. Based on cell value could know if slave or master on that system.
      • Stefano: That might work too
      • Rob: Assuming address range might break if >1 reg entry, generically
      • Stefano: Different slave ports on different reg entries?
    • Stefano: Assumption: Each node, even if it has multiple regs, are all slave the same way & master the same way. No different master & slave config for each reg entry of 1 node. Think it holds true on our boards.
      • Rob: Cell values could be address, even if duplicated. May be a subset of reg entries.
      • Stefano: Good idea. Could put it as a 1st ID after lpd_xppu
      • Rob: Generally cells only completely meaningful to phandle they are associated w/. If common, would be convenient.
    • Stefano: Will try to combine the 2 properties into 1 & come up with some examples.
Add a custom footer
Add a custom sidebar
Clone this wiki locally