-
Notifications
You must be signed in to change notification settings - Fork 19
Asset Indexation FAQ
The goal is this article is to give a good explanation of how the Asset Indexation works, why we have such a system and how you can tweak your indexation settings to get the most out of the Search Workflow. Warning: this is about to get deep. Real Deep.
The Search Window has started way back in 2019-2020 with the QuickSearch package. Our goal with this package was to have a Apple Spotlight like workflow where you could easily search ANYTHING within Unity. Not just Assets like the Project Browser does. Not just Hierarchy GameObjects. ANYTHING. Menus, Shortcuts, Settings, Static API. We created the QuickSearch workflow so users can register SearchProvider that can yield any type of items in any way they want: synchronous search, external processes search (think Entrian), REST Calls. The way a SearchProvider would yield items wasn't important. What was important was to have a quick way to
- Open the Search Window (Ctrl + K)
- Enter a search query (focus should already be in the SearchField)
- Select an item using the keyboard
- Act on this item using SearchAction.
At first all our SearcgProvider would use already existing Unity APIs to yield thos items:
- Project => was using AssetDatabase.FindAssets
- Hierarchy: was walking each scene objects
- Menu: was using an internal API to get all menu items
We noticed that for big project using AssetDatabase.FindAssets
wasn't always as quick as we like. To be honest since 2019 the AssetDatabase has been improved tremendously and is now way faster than before. Kudos to the whole ADB team!
To ensure that we could search as fast as possible to ensure that any search could be done asynchronously we decided to add an Indexing workflow to Search. This indexation had to happened asyncornously to not block the user from interacting with Unity and should allow fast RETRIEVING of datas. We built the whole system so getting items would be as fast as possible.
When assets are imported, we register a custom asset indexer that runs in an external process to the editor: this process is called the AssetWorker process.
During indexation we generate artifacts corresponding to all searchable properties of an asset. When indexation is finalize, we combine all these artifacts into a single binary blob located in <Project>/Library/Search
. These binary blod are optimized for fast retrieving of information. The Combine steps happens in the Editor but is threaded so it shouldn't block the UI.
When assets change on disk, we do incremental indexation and recombine our artifacts into those big blobs. All of this process happens automatically and should be transparent. During these indexation steps you will see these entries in the Progress window:
When you first open the Search Window in a new project, we create a default .index file in your UserSettings folder. This default index define how indexation should be performed.
Normally a project should contain a single index file, but for specific workflows it might be good to create multiple ones (more on this later). All indexing settings can be tweaked trough the Index Manager:
An Index file by itself is really simple. It is a json file that encapsulate all indexing settings:
{
"name": "Assets",
"roots": ["Assets"],
"includes": [],
"excludes": [],
"options": {
"types": true,
"properties": true,
"extended": false,
"dependencies": false
},
"baseScore": 999
}
We have a great documentation page for the Index Manager in the official Unity Manual.
Indexation time is linear to the amount of assets in your projects but it also depends on how much data you want to index:
-
Types: Index all type information for an asset allowing the use of the
t:
filter. It should always be checked. -
Properties: this option allow for powerful search by properties workflow where you can query for any property values with various operatos (<. :,=, >...). That said, indexing ALL SerializedProperty of all assets in your project can be really long. I would suggest that for big project you disabled this option. I will give you more tips on how to do some limited search by properties later.
-
Sub objects (or extended in 23.X): this means that all subobjects of any assets will also be indexed. Concretely: all sub object of prefabs and scenes will be indexed. This allows for offline search of scene. But this comes at a high cost because scene can contains LOTS of objects. I suggest that for medium or big project this option should be disabled.
-
Dependencies: this uses AssetDatabase.GetDependencies to index asset references. I feel this option is really powerful and shines if you want to use the
ref
keyword to find all sorts of asset relationship. It should be enabled for most projects.
If you want to be able to search for assets using the ref:
filter, the Dependencies option must be toggled.
p: ref="Assets/Editor/Content/test_material_42.mat"
If you want to query asset referencing using specific properties you need to enable the Properties option:
p: t:SearchContextAttributeTest materialnosearchbar="Assets/Editor/Content/test_material_42.mat"
This whole article started when I read this post on the forums. The user liked the idea of using the Search Window but thought indexing was too long. Here are a couple of tips to improve indexing performance for your project:
-
Tweak your [Indexation settings](Indexation settings) correctly. If you feel property indexing is not something you are looking for disable it (more on this below).
-
Tweak which folder you want to include or exclude during indexation. Often times, your project can have bunch of files that are not useful to search for or to index: psd, audio, video. Only keep what gives value to your search. Especially if you index all properties.
-
Did you know you can have multiple indices in your project? And that each index file can have its own indexing settings? If you know you want to index properties for materials and prefabs but not for any other assets (ex: textures) create multiple indices each with its own settings. Then using the Asset SearchProvider we will aggregate results coming from multiple indices and the whole process will be transparent.
Alternatively if you want to force a specific search to happens on a specific index you can use the a:
filter. In this example TestPrefabs is the name of an index in my project:
p: a:TestPrefabs cu
If you like the concept of property indexing but your project is too big or you only have a small subset of properties you would like to be indexed, you can use our Custom Indexation workflow.
The CustomObjectIndexerAttribute allows to add custom property indexing to some specific type of assets.
I have written an article on how you can do Custom Indexing feel free to dig into it for more Search goodness.
Let's say you want to write your own SearchProvider with its own set of data that would warrant a fast search mechanism. Or you are building an editor workflow that would benefit from a quasi-database of properties that are fast to retrieve. Look no further than ou Search building blocks:
- QueryEngine: this class allows you to parse a search string, tokenize it and apply search operator to it : filters, boolean operators, parathesis grouping, transform operators. It can be as simple or as complex as you like. We use this class internally to parse all sorts of query string.
- SearchIndexer and ObjectIndexer: these classes allow you to add properties and words to an index and do fast retrieving of information through a query string.
You feel I have failed you. Indexing is too slow for your taste. Or you do not need anything more than the Classic Unity Search powered by the AssetDatabase (like the one provided by the Project Browser) but somehow you like the Search Window as the dedicated hb for all your search. We still have you covered.
Did you know you can disable and enable any Search Providers in the Search Window and that those providers will stay enabled/disabled each time you open the Search Window?
If ADB is enough for you then use it through the Search Window. Disable the Project provider and keep ADB. Then use the ADB Search provider for all your searches.
You can combine adb provider and the table workflow:
You can also persist on disk any Queries involving the ADB provider. For more information on SearchQueryAsset, Table configuration and the power of Collection see this article.
I hope this article helped you better understand how the indexing process is triggered and managed in Unity and how you can improve its performance by tweaking your indices. If all else fails do not bail on the Search Window and all its excellent workflows and use the ADB provider.