- PEP 695 support
- parser: PEP 695 - Type Parameter Syntax #1004
- Scope provider: support for type annotations #1014
- PEP 701 support
- parser: support arbitrarily nested f-strings #1026
- parser: Parse multiline expressions in f-strings #1027
- parser: Support files with mixed newlines #1007
- libcst is now published to crates.io
- codemod/ApplyTypeAnnotationsVisitor: Do not annotate the same variable multiple times #956
- parser: Don't swallow trailing whitespace #976
- codemod/rename: Avoid duplicating import statements when the module name doesn't change #981
- cli: Don't gather dirs ending .py #994
- drop support for Python 3.7 #997
- A few parser performance improvements:
- Switch to using thread_local regular expressions to stop mutext contention #996
- Remove need for regex in TextPosition::matches #1002
- Remove Regexes from whitespace parser #1008
- Fix type of
evaluated_value
on string to allow bytes by @ljodal in Instagram#721 - Fix Sentinal typo by @kit1980 in Instagram#948
- Allow no whitespace after lambda body in certain cases by @zsol in Instagram#939
- Fix whitespace, fstring, walrus related parse errors (#939, #938, #937, #936, #935, #934, #933, #932, #931) by @zsol in Instagram#940
- Codemod CLI: Print diff only when there is a change by @kit1980 in Instagram#945
- @ljodal made their first contribution in Instagram#721
- @kit1980 made their first contribution in Instagram#948
Full Changelog: https://github.com/Instagram/LibCST/compare/v1.0.0...v1.0.1
The first major release of LibCST is essentially the same as 0.4.10, but using the
newer, Rust-based parser implementation by default. The old, pure Python parser is
scheduled for removal in the next (non-patch) release. Until then, it is available with
the LIBCST_PARSER_TYPE
environment variable set to pure
.
- Switch the default parser implementation to native by @zsol in Instagram#929
- @and-semakin made their first contribution in Instagram#816
- @carljm made their first contribution in Instagram#828
- @sagarbadiyani made their first contribution in Instagram#841
- @podtserkovskiy made their first contribution in Instagram#894
- @rchen152 made their first contribution in Instagram#903
- @Kludex made their first contribution in Instagram#913
- @jakkdl made their first contribution in Instagram#921
- Add py3.11 classifier by @and-semakin in Instagram#816
- Script to regenerate test fixtures, upgrade to Pyre 0.9.10 by @amyreese in Instagram#872
- Allow FullyQualifiedNameProvider to work with absolute paths by @amyreese in Instagram#867
- Allow running codemods without configuring in YAML by @akx in Instagram#879
- Support PEP 604 in ApplyTypeAnnotationsVisitor by @hauntsaninja in Instagram#868
- fix PEP 604 union annotations in decorators by @carljm in Instagram#828
- [AddImportsVisitor] Docstring Check Only for the Top Element of the Body by @sagarbadiyani in Instagram#841
- Fix #855 - fail to parse with statement by @stroxler in Instagram#861
- Add setuptools-rust to build requirements in setup.py by @amyreese in Instagram#873
- Relative imports from '' package are not allowed by @podtserkovskiy in Instagram#894
- Use subprocess.DEVNULL instead of opening os.devnull by hand by @akx in Instagram#897
- Ensure current Python interpreter is used for subprocesses by @akx in Instagram#898
- Fix ApplyTypeAnnotationsVisitor behavior on attribute assignments. by @rchen152 in Instagram#903
- Fix spelling and grammar in some comments by @stroxler in Instagram#908
- skip escaped backslash in rf-string by @jakkdl in Instagram#921
- relax validation rules on decorators by @jakkdl in Instagram#926
Full Changelog: https://github.com/Instagram/LibCST/compare/v0.4.9...v0.4.10
- Bump setuptools-rust version by @zsol in Instagram#809
Full Changelog: https://github.com/Instagram/LibCST/compare/v0.4.8...v0.4.9
- @dhruvmanila made their first contribution in Instagram#728
- @vfazio made their first contribution in Instagram#801
- @matthewshaer made their first contribution in Instagram#807
- Fix parse error message for number parsing by @zzl0 in Instagram#724
- Fix problematic doc build, due to the new builder image provided by readthedocs doesn't has the
graphviz-dev
package pre-installed any more by @MapleCCC in Instagram#751 - Fix docstring of
FullRepoManager
by @MapleCCC in Instagram#750 - Fix bug when
TypeOf
is one of options inOneOf
/AllOf
by @MapleCCC in Instagram#756 - Tighten the metadata type of
ExpressionContextProvider
by @MapleCCC in Instagram#760 - Fix the bug that the use of formatter in codemods has undetermined target Python version, resulting in hard-to-reason-with behavior by @MapleCCC in Instagram#771
- Python 3.11 rutime support
- test using python 3.11 beta versions by @zsol in Instagram#723
- Python 3.11 wheels by @vfazio in Instagram#801
- Raise informative exception when metadata is unresolved in a metadata-based match by @MapleCCC in Instagram#757
- Add AccessorProvider by @matthewshaer in Instagram#807
Full Changelog: https://github.com/Instagram/LibCST/compare/v0.4.7...v0.4.8
- @Chenguang-Zhu made their first contribution in Instagram#720
- Fix get_qualified_names_for matching on prefixes of the given name by @lpetre in Instagram#719
- Implement lazy loading mechanism for expensive metadata providers by @Chenguang-Zhu in Instagram#720
- @superbobry made their first contribution in Instagram#702
- convert_type_comments now preserves comments following type comments by @superbobry in Instagram#702
- QualifiedNameProvider optimizations
- Cache the scope name prefix to prevent scope traversal in a tight loop by @lpetre in Instagram#708
- Faster qualified name formatting by @lpetre in Instagram#710
- Prevent unnecessary work in Scope.get_qualified_names_for_ by @lpetre in Instagram#709
- Fix parsing of parenthesized empty tuples by @zsol in Instagram#712
- Support whitespace after ParamSlash by @zsol in Instagram#713
- [parser] bail on deeply nested expressions by @zsol in Instagram#718
- @zzl0 made their first contribution in Instagram#704
- Only skip supported escaped characters in f-strings by @zsol in Instagram#700
- Escaping quote characters in raw string literals causes a tokenizer error by @zsol in Instagram#668
- Corrected a code example in the documentation by @zzl0 in Instagram#703
- Handle multiline strings that start with quotes by @zzl0 in Instagram#704
- Fixed a performance regression in libcst.metadata.ScopeProvider by @lpetre in Instagram#698
- @adamchainz made their first contribution in Instagram#688
- Add package links to PyPI by @adamchainz in Instagram#688
- native: add overall benchmark by @zsol in Instagram#692
- Add support for PEP-646 by @zsol in Instagram#696
- parser: use references instead of smart pointers for Tokens by @zsol in Instagram#691
- Restore the 0.4.1 behavior for libcst.helpers.get_absolute_module by @lpetre in Instagram#684
- @stanislavlevin made their first contribution in Instagram#650
- @dmitryvinn made their first contribution in Instagram#655
- @wiyr made their first contribution in Instagram#669
- @toofar made their first contribution in Instagram#675
- native: Avoid crashing by making IntoPy conversion fallible by @zsol in Instagram#639
- native: make sure ParserError's line is zero-indexed by @zsol in Instagram#681
- Fix space validation for AsName and Await by @zsol in Instagram#641
- Qualified Name Provider: Fix returned qname for symbols that are prefixes of each other by @wiyr in Instagram#669
- Rename Codemod: Correct last renamed import from by @toofar in Instagram#675
- Many changes to the Apply Type Comments codemod:
- Allow for skipping quotes when applying type comments by @stroxler in Instagram#644
- Port pyre fixes by @stroxler in Instagram#651
- Preserve as-imports when merging type annotations. by @martindemello in Instagram#664
- Qualify imported symbols when the dequalified form would cause a conflict by @martindemello in Instagram#674
- Add an argument to always qualify imported type annotations. by @martindemello in Instagram#676
- Create an AddTrailingCommas codemod by @stroxler in Instagram#643
- Define gather global names visitor by @shannonzhu in Instagram#657
- Support module and package names in the codemod context by @lpetre in Instagram#662
- Drop support for running libcst using a python 3.6 interpreter by @lpetre in Instagram#663
- Update relative import logic to match cpython by @lpetre in Instagram#660
- Scope Provider: Consider access information when computing qualified names for nodes by @lpetre in Instagram#682
- @ariebovenberg made their first contribution in Instagram#605
- @sehz made their first contribution in Instagram#598
- Add docs about the native parts by @zsol in Instagram#601
- Specify minimum rust toolchain version by @zsol in Instagram#614
- build wheels on main branch for linux/arm64 by @zsol in Instagram#630
- ApplyTypeAnnotationVisitor changes
- Add support for methods with func type comment excluding self/cls by @stroxler in Instagram#622
- Merge in TypeVars and Generic base classes in ApplyTypeAnnotationVisitor by @martindemello in Instagram#596
- Full handling for applying type comments to Assign by @stroxler in Instagram#599
- Add support for For and With by @stroxler in Instagram#607
- Support FunctionDef transformations by @stroxler in Instagram#610
- change pyo3 as optional dependency in native Python Parser by @sehz in Instagram#598
- add slots to base classes, @add_slots takes bases into account by @ariebovenberg in Instagram#605
- [native] Box most enums by @zsol in Instagram#632
- [native] Return tuples instead of lists in CST nodes by @zsol in Instagram#631
- Allow trailing whitespace without newline at EOF by @zsol in Instagram#611
- Handle ast.parse failures when converting function type comments by @stroxler in Instagram#616
- [native] Don't redundantly nest StarredElement inside another Element by @isidentical in Instagram#624
- [native] Allow unparenthesized tuples inside f-strings by @isidentical in Instagram#621
- Don't require whitespace right after match by @isidentical in Instagram#628
- Proxy both parentheses in some pattern matching nodes by @isidentical in Instagram#626
This release contains a new parsing infrastructure that is turned off by default. You
can enable it by setting the LIBCST_PARSER_TYPE
environment variable to native
before parsing an input with the usual LibCST APIs. Parsing Python 3.10 documents is
only supported in this new mode.
Note: the new parser is built as a native extension, so LibCST will ship with binary wheels from now on.
- Implement a Python PEG parser in Rust by @zsol in #566
- implement PEP-654: except* by @zsol in #571
- Implement PEP-634 - Match statement by @zsol in #568
- Add instructions to codegen test failures by @stroxler in #582
- Support Parenthesized With Statements by @stroxler in #584
- Support relative imports in AddImportsVisitor by @martindemello in #585
- Codemod for PEP 484 Assign w / type comments -> PEP 526 AnnAssign by @stroxler in #594
- Update license headers by @zsol in #560
- Use precise signature matching when inserting function type annotations by @martindemello in #591
- Fix missing string annotation references #561
- Add --indent-string option to
libcst.tool print
#525 - Publish pre-release packages to test.pypi.org #550
- Add ImportAssignment class extending Assignment to record assignments for import statements #554
- Various documentation fixes #527, #529
- Do not add imports if we added no type info in ApplyTypeAnnotationVisitor (commit)
- Support relative imports in ApplyTypeAnnotationVisitor qualifier handling #538
- Don't gather metadata if the wrapper already contains it #545
- Swallow parsing errors in string annotations #548
- Stop parsing string annotations when no longer in a typing call #546
- Move find_qualified_names_for in the Assignment class #557
- Fix pyre command for type inference provider #523
- Change codegen to treat typing.Union[Foo, NoneType] and typing.Optional[Foo] as the same [#508]((Instagram#508)
- Rewrite the MatchIfTrue type to be generic on _MatchIfTrueT #512
- Add python3.9 to the CI #506
- Various CI changes #471 #510 #505 #515 #516
- Don't reset subprocess environment to fix codemodding on windows #495
- TypeAnnotationsVisitor: don't truncate function return type #499
- Docs: Fix typo #492
- Return more specific QNames for assignments #477
- Tie accesses from string annotation to the string node #483
- Fix leaking processes from TypeInferenceProvider #474
- Fix TypeInferenceProvider breakage with empty cache #476
- Fix formatting for link to QualifiedName class in docs #480
- Add FlattenSentinel to support replacing a statement with multiple statements #455
- Add BuiltinScope #469
- Add FullyQualifiedNameProvider #465
- Split QualifiedNameProvider out from libcst.metadata.scope_provider #464
- Exception while parsing escape character in raw f-strings #462
- Optimization: reduce the number of unused parallel processes #440
- Walrus operator's left hand side now has STORE expression context #443
- ApplyTypeAnnotationsVisitor applies parameter annotations even if no return type is declared #445
- Work around Windows problem by using dummy pool for
jobs=1
#436 - Remove extra unused imports added in other files #453
- Support PEP-604 style unions in decorator annotations #429
- Gathering exports in augmented assignment statements #426
- Don't allow out of order accesses in the global scope #431
- Handle scope ordering in For statements #430
- Fix for not parsing subscripts such as
cast()["from"]
#428 - Walrus operator's left hand side now has STORE expression context #433
- Support Named Unicode Characters and yield in f-strings #424
- Fix is_annotation for types used in classdef base and assign value #406
- Visit concatenated f-strings during scope analysis #411
- Correct handling of walrus operator in function args #417
- Allow generator expressions in f-strings #419
- Keep track of assignment/access ordering during scope analysis #413
- Handle string type references in cast() during scope analysis #418
- Use correct type for AugAssign and AnnAssign target #396
- Support string annotations for type aliases #401
- fix RemoveImportsVisitor crash when ImportAlias is inserted without comma #397
- Provide STORE for {Class,Function}Def.name in ExpressionContextProvider #394
- Implement TypeOf matcher #384
- Update return type of ParentNodeProvider to be CSTNode #377
- Add source code links to each class/function #378
- Removing an import alias with a trailing standalone comment should preserve the comment #392
- Handle string annotations in ScopeProvider #373
- Add is_annotation subtype for Access inreferences. #372
- Call pyre query with noninteractive logging #371
- Replace matchers with explicit visitation in gatherers #366
- Include missing test data in install #365
- Spaces around walrus operator are not required #368
- SaveMachedNode now matches with trailing empty wildcards #356
- Correctly extract wildcard matchers #355
- Support string type annotations in RemoveUnusedImports #353
- Add scope to ImportAlias #350
- Add scope to ClassDef #349
- Handle type subscripts when applying annotations. #335
- Added FullRepoManager
cache
property #330 - Added optional args for tox commands #327
- Only remove trailing comma if the last alias is removed #334
- Fixed inserting imports after module docstring #343
- Fixed ParenthesizedWhitespace before params in FuncDef #342
- Fixed validation for ImportAlias and Try statements #340
- Fixed NotEqual position issue #325
- Fixed minor typo in scope_provider.py #324
- Added
RenameCommand
to rename all instances of a local or imported object to a specified new name. #308
- Fixed
ImportError
under Python 3.9. #306 - Fixed
stdout
being plugged into successfully codemod-ed files. #309 - Fixed
QualifiedName
retrieval for names with repeated substrings. #312 - Fixed default values of keyword-only and positional-only arguments in
ApplyTypeAnnotationsVisitor
. #314 - Fixed
ExpressionContextProvider
by giving subscript values aLOAD
context. #319
- Added
ConvertNamedTupleToDataclassCommand
to convertNamedTuple
class declarations to Python 3.7dataclasses
using the@dataclass(frozen=True)
decorator. #299
- Fixed typo in file name
libcst/codemod/commands/convert_percent_format_to_fstring.py
. #301 - Fixed
StopIteration
exception during scope analysis matching on import names. #302
- Expose more granular
Assignments
andAccesses
for dotted imports inScopeProvider
. #284 get_qualified_names_for
returns the most appropriate qualified name. #290- Surface
SyntaxError
raised by formatter in codemod run. #288 #289 - Rename
ApplyTypeAnnotationsVisitor.add_stub_to_context
asApplyTypeAnnotationsVisitor.store_stub_in_context
and addoverwrite_existing_annotations
to allow overwrite existing type annotations. #289
- Close opened file handles on finishing codemod to avoid
Too many open files
on OSX. #283
ApplyTypeAnnotationsVisitor.add_stub_to_context
is renamed asApplyTypeAnnotationsVisitor.store_stub_in_context
.
- Supported CST parsing for Python 3.0, 3.1 and 3.3. #261
- Added
RemoveUnusedImportsCommand
for removing unused import codemod. #266 - Added
ApplyTypeAnnotationsVisitor.add_stub_to_context
for apply type annotations from stub modules. #265
- Improved exception message of
get_metadata
when MetadataWrapper is not used. #257 - New steps for Pyre type check in README.rst which analyzes installed Python sources for better type checking. #262
- Parsed
except(Exception):
correctly while there is no space after except syntax. #256 - Fixed
RemoveImportsVisitor
to not remove imports when references still exist. #264 - Fixed missing type annotations. #271
AddImportsVisitor
generates deterministic order for added imports. #274
ByteSpanPositionProvider
provides start offset and length of CSTNode as metadata.get_docstring
helper provides docstring fromModule
,ClassDef
andFunctionDef
node types.
- Optimized
ScopeProvider
performance to run faster and use less memory:- remove unnecessary
Assignment
of keywordArg
. - don't provide scope object for formatting information nodes.
- batch set union updates in
infer_accesses
step.
- remove unnecessary
- Fixed
_assignments
mutation when calling read-onlyScope.get_qualified_names_for
and__contains__
.
- Added
RemoveImportsVisitor
to remove an import if it's not used in a module. - Added
GatherExportsVisitor
to gather exports specified in__all__
. - Added property helpers
evaluated_name
andevaluated_name
inImportAlias
. - Added helper to get full module name:
get_absolute_module_for_import
andget_absolute_module_for_import_or_raise
. - Added
CodemodContext.full_module_name
for full dotted module name. - Added format specifiers f-string conversion support to
ConvertFormatStringCommand
.
- Moved LibCST version to
_version.py
and can print it bypython -m libcst.tool --version
. - Improved
EnsureImportPresentCommand
with--alias
option. - Improved
ConvertFormatStringCommand
with--allow-strip-comments
and--allow-await
options.
- Added helpers to get both the raw and evaluated value of a SimpleString.
- Added helpers to get the quoting and prefix of SimpleString and FormattedString.
- Added a helper to get the evaluated value of number types.
- Added templated parsers for statement/expression/module to make constructing updated nodes in transforms easier.
- FullRepoManager is now integrated into codemods, so metadata requiring full repo analysis can now be used.
- Added
get_full_name_for_node_or_raise
helper to remove boilerplate of checking againstNone
.
- Upgraded Pyre dependency to 0.0.41.
- Added additional status to
libcst codemod
command. get_full_name_for_node
now supports decorators.
- Clarified documentation around f-strings, fixed indentation.
- Fixed
libcst list
crashing if a codemod does unsafe work on import. - Fixed deploy-time dependencies so pyyaml won't have to be manually installed to execute codemods.
- QualifiedNameProvider no longer erroneously claims names inside attributes are built-ins.
- Added support for parsing and rendering Python 3.8 source code.
- Added more documentation for codemods.
- Added
get_full_name_for_expression
helper method. - Added
has_name
helper toQualifiedNameProvider
. - Added a
--python-version
flag tolibcst.tool print
utility.
- Codemod command can now discover codemods in subdirectories of configured modules.
- Updgraded Pyre dependency to 0.0.39.
- Cleaned up some typos and formatting issues in comments and documentation.
- Cleaned up a few redundant typevars.
- Fixed callable typing in matchers implementation.
- Fixed incorrect base class references in matcher decorator attribute visitors.
- Fixed codemod test assertion failing for some whitespace edge cases.
- Fixed scope analysis to track variable usage on
del
statements.
- Deprecated exporting
ensure_type
fromlibcst
in favor oflibcst.helpers
.
- Removed
ExtSlice
and helper code in favor ofSubscriptElement
. - Removed
default_params
attribute onParameters
. - Removed
SyntacticPositionProvider
andBasicPositionProvider
. - Removed
CodePosition
andCodeRange
exports onlibcst
in favor oflibcst.metadata
.
- Command-line interface now shows rough estimate of time remaining while executing a codemod.
- Add needed import now supports import aliases.
- Added Codemod framework for running code transform over a codebase in parallel.
- Codemod for code transform logic.
- CodemodContext for preserving states across transforms.
- CodemodCommand for CLI interface.
- CodemodTest for testing codemod easily.
- yaml codemod config.
- Pre-build commands in codemod/commands/.
- Added TypeInferenceProvider for inferred type info from Pyre. A regression test suite was included.
- Added FullRepoManager for metadata inter-process cache handing.
- Fixed usage link in README.
- Fixed type annotation for Mypy compatibility.
- Upgraded Pyre to 0.0.38
- Added
extract
,extractall
andreplace
functions to Matchers API.
- Fixed length restrictions for
AllOf
andOneOf
so that they can be used with sequence expansion operators. - Fixed batchable visitors not calling attribute visit functions.
- Fixed typos in docstrings.
- Fixed matcher type exception not being pickleable.
- Deprecated parsing function parameters with defaults into
default_params
attribute. They can be found in theparams
attribute instead.
- Fixed broken types for sequence matchers.
- Preliminary support for 3.8 walrus operator.
- CI config and fuzz tests for 3.8.
- Experimental re-entrant codegen API.
- Added
unsafe_skip_copy
optimization toMetadataWrapper
. - Matchers API now includes a
findall
function. - Matchers now have a
MatchMetadataIfTrue
special matcher.
- Updated to latest Black release.
- Better type documentation for generated matchers.
- Clarified matchers documentation in several confusing areas.
- Drastically sped up codegen and tests.
QualifiedName
now supports imported attributtes.ExpressionContext
properly marks loop variables asSTORE
.- Various typos in documentation are fixed.
- Deprecated
BasicPositionProvider
andSyntacticPositionProvider
in favor ofWhitespaceInclusivePositionProvider
andPositionProvider
.
- Added
deep_with_changes
helper method on CSTNode. - Added metadata support to matchers.
- Added ability to get the defining node from a
LocalScope
(FunctionScope
,ClassScope
orComprehensionScope
).
- Many changes to LibCST documentation including a new best practices page and updated scope tutorial.
- Exported
CodePosition
andCodeRange
fromlibcst.metadata
instead oflibcst
.
- Disallowed decorating a concrete visit or leave method with
@visit
or@leave
decorators. - Renamed position provider classes to be more self-explanatory.
- Fixed trailing newline detection when the last character in a file was from a continuation.
- Fixed
deep_clone
potentially blowing the stack with large LibCST trees.
- Deprecated
ExtSlice
in favor ofSubscriptElement
. - Deprecated parsing
Subscript
slices directly intoIndex
orSlice
nodes.
Scope.assignments
andScope.accesses
APIs to access all references in a scope.- Scope analysis tutorial.
- Supported
<comprehension>
inScope.get_qualified_names_for
andQualifiedName
. - Enforced identity equality for matchers and immutability of non-dataclass matchers.
- Generalize codegen cleanup steps for all codegen.
- Made
BatchableMetadataProvider
typing covariant over its typevar. - Fixed LICENSE header on generated matcher file.
- Cleanup unused internal noqa and on-call specification.
- Added matchers which allow comparing LibCST trees against arbitrary patterns.
- Improved tree manipulation with
deep_remove
anddeep_replace
helper methods on CSTNode. - Added new metadata providers: parent node and qualified name.
- Updated Pyre to latest release.
- Updated scope metadata to provide additional helpers.
- Updated preferred method of removing a node from its parent in a visitor.
- Metadata classes and methods are now exported from "libcst.metadata" instead of several submodules.
- Fixed LICENSE file to explicitly reference individual files in the repo with different licenses.
- Fixed
deep_clone
to correctly clone leaf nodes. - Fixed all parse entrypoints to always return a tree with no duplicated leaf nodes.
- Added preliminary support for parsing Python 3.5 and Python 3.6 source.
- Added scope analysis metadata provider.
- Added mypy type support for built package.
- Several typos in documentation are fixed.
- Added attribute visitor hooks.
- Added base visit/leave methods which can be subclassed.
- Hypothesis fuzz testing suite, courtesy of Zac Hatfield-Dodds.
- Metadata documentation is much more complete.
- Fixed several whitespace validation issues caught by Hypothesis.
- Parser syntax errors are now used inside parser.
- Metadata interface is now exported.
- Dependencies are now specified with minimum revisions.
- Lots of documentation fixes.
- First public release of LibCST.
- Complete, fully typed syntax tree for Python 3.6.
- Full suite of tests for each defined node type.