On November 20, 2019, at FINOS' flagship conference, the Open Source Strategy Forum, FINOS Platinum member Goldman Sachs announced its intention to open source two closely related products into FINOS:
- Its internally developed logical modeling language, referred to within the bank as "PURE", and
- A workbench of tools, including a visual modeling platform, collectively referred to as "Alloy", created and used within Goldman Sachs to build, design, and execute PURE models
The PURE language, together with four modules - Studio, SDLC, Engine, and Shared - were open sourced into FINOS on October 19, 2020. Concurrent to their open sourcing, they were collectively given a new, unified name - "Legend". The choice of the Legend name reflects how for Goldman Sachs the language and platform have been a guide - a legend - for the bank's data strategy.
A shared, external instance of Legend deployed on AWS within FINOS' infrastructure is available at https://legend.finos.org/studio. The primary purpose of this instance is to provide a shared modeling environment for the FINOS community and its projects. An additional purpose of this instance is to provide a place where people can try out Legend for themselves, check out its tools and features, and see what a Legend model looks like. This instance of Legend was the same instance that was used by the pilot group to model in Phase 1 (see Roadmap section below for more info on our phases).
To request an account on the shared instance, use the form at the bottom of https://www.finos.org/legend. You will need a GitHub account and, if want to be able to create and edit models, be covered by a Contributor License Agreement with FINOS (otherwise a read-only account will be created).
Note that the features available in the shared, external Legend instance within FINOS infrastructure largely map to the five modules open sourced to date. Features that are not yet available include model-to-database mappings and the execution of queries as services. As more modules are open sourced, more features will be activated in the shared instance.
(Coming Soon; Information on running and installing your own instance of Legend, locallly, on premise, or in a cloud provider such as AWS, Azure, OpenShift, Google, etc.)
The Legend platform and project makes use of both GitHub and GitLab.
- GitHub, specifically the FINOS GitHub organization, is the distributed source and version control management system used for Legend code.
- The Legend GitHub repos, like all FINOS repos, are configured to make use of the FINOS Open Developer Platform tooling.
- GitHub infrastructure is what's used to do Legend builds (e.g., of Legend JARS, which are then distributed through Maven Central)
- GitLab is used for source control of models "under the hood" within the Legend SDLC module. As such the models, written in the Legend language, are stored in GitLab repositories.
This repository (finos/legend) is the parent repository for the project. It is used principally for overall Legend documentation and issue tracking. The code itself for the 5 modules open sourced thusfar can be found in the following 5 child repositories:
- Shared Module: https://github.com/finos/legend-shared
- SDLC Module: https://github.com/finos/legend-sdlc
- Studio Module: https://github.com/finos/legend-studio
- Engine Module: https://github.com/finos/legend-engine
- Legend Language: https://github.com/finos/legend-pure
Module specific issues are opened, maintained, and prioritized within the modules specific issue list (e.g., https://github.com/finos/legend-studio/issues for Legend Studio). Issues can be aggregated from multiple reposistories as needed through FINOS organization level GitHub kanban boards("Projects").
Legend models developed on the external shared instance, including those built during the Phase 1 pilot described below, are themselves open sourced code and available in the following GitLab repositories:
- FX Options extensions to the CDM: https://gitlab.legend.finos.org/legend-pilot/cdm
- Commodities Reference Data: https://gitlab.legend.finos.org/legend-pilot/commonrefdata
Note on model access in GitLab Currently to access (and fork, etc.) these GitLab model repositories directly, you will need an account on the Legend shared instance (an account is also required to model and access these models through the Studio UI). An account may be requested through the account request form at the bottom of https://www.finos.org/legend. In the future we plan to migrate these model repositories to public repos on GitLab.com, after which an account will no longer be needed to access and contain the repositories that contain the models.
To learn more about how to get involved with Legend and contribute to it, see the CONTRIBUTING.md file or the Contribute section of the docs site.
(Coming Soon)
- Announced the intention to open source PURE/Alloy (now named Legend) at the Open Source Strategy Forum
- Deployed an external instance of Legend to AWS within FINOS infrastructure, initially for use in Phase 1 (see below) for pilot participants to use to do shared modeling.
- Commenced internal code evaluation and preparation for open sourcing
-
Opened the external instance of Legend Studio deployed in Phase 0 to a pilot group of financial institutions, composed of FINOS members plus several other institutions that expressed an interest in participating in the pilot, for them together with Goldman Sachs and FINOS to evaluate the use of Legend to do shared, collaborative modeling. In this release, Studio supported a limited set of functionality, including the creation of data models, model-to-model mappings, constraints and derivations
-
During the pilot phase, modelers collaborated on the Legend external instance to build shared models. Modeling occurred in two work streams:
- An FX workstream of the the pilot group extended and expanded the FX Option model of the Common Domain Model (CDM), developed by the International Swaps and Derivatives Association (ISDA). The Averaging Model developed during the pilot has since been approved by the ISDA Architecture Review Committee and released in the CDM (see 2.82.8).
- The pilot group also created a Commodity Reference Data workstream, to work on commodities data, especially in the context of a Fixed vs Float swap, including defining a payout model to be proposed into the CDM.
Other areas that the pilot group have discussed as potential areas to model together in Legend include Environmental, Social and Governance (ESG) data as well as potentially continuing some of the work started by the FINOS Financial Objects program on RFQ in the context of interest rate swaps.
-
Performed code readiness work to extract Legend code from GS infrastructure, and sufficiently abstract and generalize code base as needed. Revised code base to reflect new name (note that some references to old "PURE" and "Alloy" name still exist). GS Engineering, together with FINOS and Linux Foundation DevOps teams, performed code scanning and remediations. Moved Goldman Sachs's daily production builds of Legend to FINOS infrastructure and public GitHub repos.
- On 2020.10.19 made underlying code bases to five Legend modules available as open source under an Apache 2.0 license
- Announced new name (Legend) and rebranded
- Identify "good first issues" for developers new to Legend
- Begin accepting pull requests of code from external (non-GS) developers.
- Conduct office hours for external developers interested in getting involved with Legend
- Make additional functionality available in Studio, including relational execution, support for flat data sources (e.g., csv) and automated code generation (e.g., JSON schema, Protobuf, AVRO).
- Transition shared model repositories from hosted GitLab instance (gitlab.legend.finos.org) to gitlab.com.
- Open source Cube UI for user-friendly tabular data exploration.
- Open source Query UI to search and access modeled data.
- Open source Services UI. This will allow managing of production data services (APIs) for repeatable and controlled data access.
The documentation site is built using Docusaurus based on the markdown files in the docs folder.
This repository (github.com/finos/legend) hosts:
- The underling files for the Legend Documentation site (composed in markdown file format and stored in the
docs/folder) - The shared issues list used for task management to track project activity around documentation, deployments, and the overall effort associated with open sourcing Legend
- These issues are organized in various kanban boards, including the overall project kanban.
A separate hosted Gitlab repository, gitlab.legend.finos.org hosts shared models that were developed by the Legend project pilot team.
gitlab.com (https://gitlab.com/finosfoundation) [Not Yet Used]
In the future we intend to migrate the models repositories from the hosted GitLab repository to https://gitlab.com/finosfoundation. For now, this is unused but will eventually be the long-term home for public models built using Legend.
- Legend Demo by Pierre de Belen of Goldman Sachs at OSSF 2019 (video)
- Replay from May 6th Legend Pilot Call Presentation - Part 1 of Open Source Fundamentals & Concepts for Legend Modeling (video)
- Replay from May 20th Legend Pilot Call Presentation - Part 2 of Open Source Fundamentals & Concepts for Legend Modeling (video)
- Legend Documentation site
- Legend meeting minutes:
- If you cannot find the answer to your question in the above materials, please raise an issue at https://github.com/finos/legend/issues
Legend general meetings are your opportunity to get updates on Legend, provide feedback on the product and the roadmap, see demos, and ask questions. General meetings happen the third Thursday of each month at 9am ET / 2pm BST. The next such general meeting is scheduled for November 19, 2020.
- Thursday, October 29, 11am ET
- WebEx: https://finos.webex.com/finos/j.php?MTID=m58790a7ebaa830f785f3e0934603372b
- Meeting number (access code): 127 301 6315
- Email [email protected] to RSVP and receive the meeting password
Copyright 2020 Goldman Sachs
Distributed under the Apache License, Version 2.0.
SPDX-License-Identifier: Apache-2.0