Asset Indexation FAQ
The goal of this article is to give a good explanation of how Asset Indexation works, why we have such a system and how you can tweak your indexation settings to get the most out of the Search Workflow. Warning: this is about to get deep. Real Deep.
The Search project has started way back in 2019-2020 with the QuickSearch package. Our goal with this package was to have a Apple Spotlight like workflow where you could easily search ANYTHING within Unity. Not just Assets like the Project Browser does. Not just Hierarchy GameObjects. ANYTHING. Menus, Shortcuts, Settings, Static API. We created the QuickSearch workflow so users can register SearchProvider that can yield any type of items in any way they want: synchronous search, external processes search (think Entrian), REST Calls. The way a SearchProvider would yield items wasn't important. What was important was to have a quick way to:
- Open the Search Window (Ctrl + K)
- Immediatly start to type a search query (focus should already be in the SearchField)
- Select an item using the keyboard
- Act on this item using SearchAction.
At first all our SearchProvider would use already existing Unity APIs to yield those items:
- Project => was using AssetDatabase.FindAssets (which is a synchronous and not thread safe api)
- Hierarchy: was walking each scene objects
- Menu: was using an internal API to get all menu items
We noticed that for big project using AssetDatabase.FindAssets wasn't always as quick as we would like. To be honest since 2019 the AssetDatabase has been improved tremendously and is now way faster than before. Kudos to the whole ADB team!
To ensure that we could search as fast as possible to ensure that any search could be done asynchronously we decided to add an Indexing workflow to Search. This indexation had to happened asynchronously to not block the user from interacting with Unity and should allow fast RETRIEVING of datas. We built the whole system so getting items would be as fast as possible.
When assets are imported, we register a custom asset indexer task that runs in an external called the AssetImportWorker (this is the process responsible for Asset Import as well as its name imply):
During indexation we generate artifacts corresponding to all searchable parameters of an asset: name, folder, age, size, type, properties... When indexation is finalized for all assets, we combine all these artifacts into a single binary blob located in <Project>/Library/Search. These binary blobs are optimized for fast retrieving of information. The Combine step happens in the Editor but is threaded so it shouldn't block the UI.
When assets change on disk, we do incremental indexation and recombine our artifacts into binary blobs. All of this process happens automatically and should be transparent. During this indexation step you will see the following entry in the Progress window:
When you first open the Search Window in a new project, we create a default index file in your UserSettings folder. This file defines how indexation should be performed.
Normally a project should contain a single index file, but for specific workflows it might be good to create multiple ones (more on this later). All indexing settings can be tweaked trough the Index Manager:
An Index file by itself is really simple. It is a json file that encapsulate all indexing settings:
{
"name": "Assets",
"roots": ["Assets"],
"includes": [],
"excludes": [],
"options": {
"types": true,
"properties": true,
"extended": false,
"dependencies": false
},
"baseScore": 999
}We have a great documentation page for the Index Manager in the official Unity Manual.
Indexation time is linear to the amount of assets in your project but it also depends on how much data you want to index:
-
Types: Index all type information for an asset allowing the use of the
t:filter. It should always be checked. -
Properties: this option allows for powerful search by properties workflow where you can query for any property values with various operators (
<, :,=,>...). That said, indexing ALL SerializedProperties of ALL assets in your project can be really long. I would suggest that for big project you disabled this option. I will give you more tips on how to do some limited search by properties later.
-
Sub objects (or extended in 23.X): this means that all objects contained in an asset will also be indexed. Concretely: all objects within prefabs and scenes will be indexed. This allows for offline search of scene. But this comes at a high cost because scene can contain LOTS of objects. I suggest that for medium or big project you disable this option.
-
Dependencies: this uses AssetDatabase.GetDependencies to index asset references. I feel this option is really powerful and shines if you want to use the ref keyword to find all sorts of asset relationships. It should be enabled for most projects.
If you want to be able to search for assets using the ref: filter: the Dependencies option must be toggled.
p: ref="Assets/Editor/Content/test_material_42.mat"
If you want to query asset referencing using specific properties you need to enable the Properties option:
p: t:SearchContextAttributeTest materialnosearchbar="Assets/Editor/Content/test_material_42.mat"
I decided to write this this article after I read this post on the forums. The user liked the idea of using the Search Window but thought indexing was too long. Here are a couple of tips to improve indexing performance for your project:
-
Tweak your Indexation settings correctly. If you feel property indexing is not something you are looking for disable it (more on this below).
-
Tweak which folders you want to include or exclude during indexation. Often times, your project can have bunch of files that are not useful to search for or to index: psd, audio, video. Only keep what gives value to your search. Especially if you index all properties.
-
Did you know you can have multiple indices in your project? And that each index file can have its own indexing settings? If you know you want to index properties for materials and prefabs but not for any other assets (ex: textures) you create multiple indices each with its own settings. The search Window will aggregate results coming from multiple indices and the whole process will be transparent.
- This means you could create an index called AllAssets.index that would have dependencies and types option.
- And another one called Properties.index for all materials and prefabs that would add the properties option.
If you want to force search to happen on a specific index you can use the a: filter. In this example TestPrefabs is the name of an index in my project:
p: a:TestPrefabs cu
If you like the concept of property indexing but your project is too big or you only have a small subset of properties you would like to be indexed, you can use our Custom Indexation workflow.
The CustomObjectIndexerAttribute allows to add general property indexing to any specific type of asset.
I have written an article on how you can do Custom Indexing. Feel free to dig into it for more Search goodness.
Let's say you want to write a SearchProvider with its own set of data that would warrant a fast search mechanism. Or you are building an editor workflow that would benefit from a database of properties that are fast to retrieve. Look no further than these two Search building blocks:
-
QueryEngine: this class allows you to parse a search string, tokenize it and apply search operator to it : filters (
ref:,t:), boolean operators (or,!), parenthesis grouping, transform operators. It can be as simple or as complex as you like. We use this class internally to parse all sorts of query string. - SearchIndexer and ObjectIndexer: these classes allow you to add properties and words to an index and do fast retrieving of information through a query string. This is what we use to store our index data and this is what you are passed as a parameter if you use the CustomObjectIndexer.
You feel I have failed you. Indexing is too slow for your taste. Or you do not need anything more than the Classic Unity Search powered by the AssetDatabase (like the one provided by the Project Browser). But somehow you like the Search Window as the dedicated hub for all your searches. We still have you covered.
Did you know you can disable or enable any Search Providers in the Search Window and that those providers will stay enabled/disabled each time you open the Search Window?
If ADB is enough for you then use it through the Search Window. Disable the Project provider and keep ADB. Then use the ADB Search provider for all your searches.
You can combine adb provider and the table workflow:
You can also persist on disk any Queries involving the ADB provider. For more information on SearchQueryAsset, Table configuration and the power of Collection see this article.
I hope this helped you get a better understanding of the indexing process: how it is triggered and managed and how you can improve its performance by tweaking your indices. If all else fails do not bail on the Search Window and all its excellent workflows and use the ADB provider.