diff --git a/PLUGIN-DESIGN.md b/PLUGIN-DESIGN.md new file mode 100644 index 000000000..1cf152c06 --- /dev/null +++ b/PLUGIN-DESIGN.md @@ -0,0 +1,376 @@ +# User Plugins Design Proposal + +## Background + +This is a design proposal for the long-standing [#55](https://github.com/rojo-rbx/rojo/issues/55) +desire to add user plugins to Rojo for things like source file transformation and instance tree +transformation. + +As discussed in [#55](https://github.com/rojo-rbx/rojo/issues/55) and as initially explored in +[#257](https://github.com/rojo-rbx/rojo/pull/257), plugins as Lua scripts seem to be a good starting +point. This concept reminded me of Rollup.js plugins. [Rollup](https://rollupjs.org/guide/en/) is a +JS bundler which performs a similar job to Rojo in the JS world by taking a number of source files +and producing a single output bundle. [Rollup +plugins](https://rollupjs.org/guide/en/#plugins-overview) are written as JS files, and passed to the +primary Rollup config file. Rollup then calls in to plugin "hooks" at special times during the +bundling process. I have found Rollup plugins to be an excellent interface for both plugin +developers and end-users and therefore I have based this proposal on their API. + +This proposal attempts to take advantage of the existing "middleware" system in Rojo, and give +plugins the opportunity to: + +- Override the default choice for which internal middleware should be used for a given file path +- Transform the contents of a file before consuming it + +## Project file changes + +Add a new top-level field to the project file format: + +- `plugins`: An array of [Plugin Descriptions](#plugin-description). + - **Optional** + - Default is `[]` + +### Plugin description + +Either a `String` value or an object with the fields: + +- `source`: A filepath to the Lua source file of the plugin or a URL to a GitHub repo, optionally + followed by an `@` character and a Git tag. + - **Required** +- `options`: Any JSON dictionary. The options that will be passed to the plugin. + - **Optional** + - Default is `{}` + +In the case that the value is just a `String`, it is interpreted as an object with the `source` +field set to its value, and `options` set to its default. + +### Example + +```json +{ + "name": "ProjectWithPlugins", + "plugins": [ + "local-plugin.lua", + "github.com/owner/remote-plugin-from-tag@v1.0.0", + "github.com/owner/remote-plugin-from-head", + { "source": "plugin-with-options.lua", "options": { "some": "option" } } + ], + ... +} +``` + +## Plugin scripts + +Plugin scripts should return a `CreatePluginFunction`: + +```luau +-- Types provided in luau format + +type PluginInstance = { + name: string, + projectDescription?: (project: ProjectDescription) -> (), + syncStart?: () -> (), + syncEnd?: () -> (), + resolve?: (id: string) -> string, + middleware?: (id: string) -> string, + load?: (id: string) -> string, +} + +-- TODO: Define properly. This is basically just the JSON converted to Lua +type ProjectDescription = { ... } + +type CreatePluginFunction = (options: {[string]: any}) -> PluginInstance +``` + +In this way, plugins have the opportunity to customize their hooks based on the options provided by +the user in the project file. + +## Plugin instance + +- `name` + - **Required**: The name of the plugin that will be used in error messages, etc. +- `projectDescription(project: ProjectDescription) -> ()` + - **Optional**: Called with a Lua representation of the current project description whenever + it has changed. +- `syncStart() -> ()` + - **Optional**: A sync has started. +- `syncEnd() -> ()` + - **Optional**: A sync has finished. +- `resolve(id: string) -> string` + - **Optional**: Takes a file path and returns a new file path that the file should be loaded + from instead. The first plugin to return a non-nil value per id wins. +- `middleware(id: string) -> string` + - **Optional**: Takes a file path and returns a snapshot middleware enum to determine how Rojo + should build the instance tree for the file. The first plugin to return a non-nil value per + id wins. +- `load(id: string) -> string` + - **Optional**: Takes a file path and returns the file contents that should be interpreted by + Rojo. The first plugin to return a non-nil value per id wins. + +## Plugin environment + +The plugin environment is created in the following way: + +1. Create a new Lua context. +1. Add a global `rojo` table which is the entry point to the [Plugin library](#plugin-library) +1. Initialize an empty `_G.plugins` table. +1. For each plugin description in the project file: + 1. Convert the plugin options from the project file from JSON to a Lua table. + 1. If the `source` field is a GitHub URL, download the plugin directory from the repo with the + specified version tag (if no tag, from the head of the default branch) into a local + `.rojo-plugins` directory with the repo identifier as its name. It is recommended that users + add `.rojo-plugins` to their `.gitignore` file. The root of the plugin will be called + `main.lua`. + 1. Load and evaluate the file contents into the Lua context to get a handle to the + `CreatePluginFunction` + 1. Call the `CreatePluginFunction` with the converted options to get a handle of the result. + 1. Push the result at the end of the `_G.plugins` table + +If at any point there is an error in the above steps, Rojo should quit with an appropriate error +message. + +## Plugin library + +Accessible via the `rojo` global, the plugin library offers helper methods for plugins: + +- `toJson(value: any) -> string` + - Converts a Lua value to a JSON string. +- `readFileAsUtf8(id: string) -> string` + - Reads the contents of a file from a file path using the VFS. Plugins should always read + files via this method rather than directly from the file system. +- `getExtension(id: string) -> boolean` + - Returns the file extension of the file path. +- `hasExtension(id: string, ext: string) -> boolean` + - Checks if a file path has the provided extension. + +## Use case analyses + +To demonstrate the effectiveness of this API, pseudo-implementations for a variety of use-cases are +shown using the API. + +### MoonScript transformation + +Requested by: + +- @Airwarfare in [#170](https://github.com/rojo-rbx/rojo/issues/170) +- @dimitriye98 in [#55](https://github.com/rojo-rbx/rojo/issues/55#issuecomment-402616429) (comment) + +```lua +local parse = require 'moonscript.parse' +local compile = require 'moonscript.compile' + +return function(options) + return { + name = "moonscript", + middleware = function(id) + if rojo.hasExtension(id, 'moon') then + if rojo.hasExtension(id, 'server.moon') then + return 'lua_server' + elseif rojo.hasExtension(id, 'client.moon') then + return 'lua_client' + else + return 'lua_module' + end + end + end, + load = function(id) + if rojo.hasExtension(id, 'moon') then + local contents = rojo.readFileAsUtf8(id) + + local tree, err = parse.string(contents) + assert(tree, err) + + local lua, err, pos = compile.tree(tree) + if not lua then error(compile.format_error(err, pos, contents)) end + + return lua + end + end + } +end +``` + +### Obfuscation/minifier transformation + +Requested by: + +- @cmumme in [#55](https://github.com/rojo-rbx/rojo/issues/55#issuecomment-794801625) (comment) +- @blake-mealey in [#382](https://github.com/rojo-rbx/rojo/issues/382) + +```lua +local minifier = require 'minifier.lua' + +return function(options) + return { + name = "minifier", + load = function(id) + if rojo.hasExtension(id, 'lua') then + local contents = rojo.readFileAsUtf8(id) + return minifier(contents) + end + end + } +end +``` + +### Markdown to Roblox rich text + +```lua +-- Convert markdown to Roblox rich text format implementation here + +return function(options) + return { + name = 'markdown-to-richtext', + middleware = function(id) + if rojo.hasExtension(id, 'md') then + return 'json_model' + end + end, + load = function(id) + if rojo.hasExtension(id, 'md') then + local contents = rojo.readFileAsUtf8(id) + + local frontmatter = parseFrontmatter(contents) + local className = frontmatter.className or 'StringValue' + local property = frontmatter.property or 'Value' + + local richText = markdownToRichText(contents) + + return rojo.toJson({ + ClassName = className, + Properties = { + [property] = richText + } + }) + end + end + } +end +``` + +### Load custom files as StringValue instances + +Requested by: + +- @rfrey-rbx in [#406](https://github.com/rojo-rbx/rojo/issues/406) +- @Quenty in [#148](https://github.com/rojo-rbx/rojo/issues/148) + +```lua +return function(options) + options.extensions = options.extensions or {} + + return { + name = 'load-as-stringvalue', + middleware = function(id) + local idExt = rojo.getExtension(id) + for _, ext in next, options.extensions do + if ext == idExt then + return 'json_model' + end + end + end, + load = function(id) + local idExt = rojo.getExtension(id) + for _, ext in next, options.extensions do + if ext == idExt then + local contents = rojo.readFileAsUtf8(id) + local jsonEncoded = contents:gsub('\r', '\\r'):gsub('\n', '\\n') + + return rojo.toJson({ + ClassName = 'StringValue', + Properties = { + Value = jsonEncoded + } + }) + end + end + end + } +end +``` + +```json +// default.project.json +{ + "plugins": [ + { "source": "load-as-stringvalue.lua", "options": { "extensions": {"md", "data.json"} }} + ] +} +``` + +### File system requires + +Requested by: + +- @blake-mealey in [#382](https://github.com/rojo-rbx/rojo/issues/382) + +```lua +-- lua parsing/writing implementation here + +return function(options) + local project = nil + + return { + name = "require-files", + projectDescription = function(newProject) + project = newProject + end, + load = function(id) + if rojo.hasExtension(id, 'lua') then + local contents = rojo.readFileAsUtf8(id) + + -- This function will look for require 'file/path' statements in the source and replace + -- them with Roblox require(instance.path) statements based on the project's configuration + -- (where certain file paths are mounted) + return replaceRequires(contents, project) + end + end + } +end +``` + + + +## Implementation priority + +This proposal could be split up into milestones without all the features being present till the end +to simplify development. Here is a proposal for the order to implement each milestone: + +1. Loading plugins from local paths +1. Minimum Rojo plugin library (`readFileAsUtf8`) +1. Calling hooks at the appropriate time + 1. Start with `middleware`, `load` + 1. Add `syncStart`, `syncEnd`, `resolve` + 1. Add `projectDescription` +1. Full Rojo plugin library +1. Loading plugins from remote repos diff --git a/src/change_processor.rs b/src/change_processor.rs index b7ad8b203..6900e1a54 100644 --- a/src/change_processor.rs +++ b/src/change_processor.rs @@ -10,6 +10,7 @@ use rbx_dom_weak::types::{Ref, Variant}; use crate::{ message_queue::MessageQueue, + plugin_env::PluginEnv, snapshot::{ apply_patch_set, compute_patch_set, AppliedPatchSet, InstigatingSource, PatchSet, RojoTree, }, @@ -49,6 +50,7 @@ impl ChangeProcessor { pub fn start( tree: Arc>, vfs: Arc, + plugin_env: Arc>, message_queue: Arc>, tree_mutation_receiver: Receiver, ) -> Self { @@ -57,6 +59,7 @@ impl ChangeProcessor { let task = JobThreadContext { tree, vfs, + plugin_env, message_queue, }; @@ -108,6 +111,8 @@ struct JobThreadContext { /// A handle to the VFS we're managing. vfs: Arc, + plugin_env: Arc>, + /// Whenever changes are applied to the DOM, we should push those changes /// into this message queue to inform any connected clients. message_queue: Arc>, @@ -125,6 +130,7 @@ impl JobThreadContext { // For a given VFS event, we might have many changes to different parts // of the tree. Calculate and apply all of these changes. let applied_patches = { + let plugin_env = self.plugin_env.lock().unwrap(); let mut tree = self.tree.lock().unwrap(); let mut applied_patches = Vec::new(); @@ -153,7 +159,9 @@ impl JobThreadContext { }; for id in affected_ids { - if let Some(patch) = compute_and_apply_changes(&mut tree, &self.vfs, id) { + if let Some(patch) = + compute_and_apply_changes(&mut tree, &self.vfs, &plugin_env, id) + { applied_patches.push(patch); } } @@ -257,7 +265,12 @@ impl JobThreadContext { } } -fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option { +fn compute_and_apply_changes( + tree: &mut RojoTree, + vfs: &Vfs, + plugin_env: &PluginEnv, + id: Ref, +) -> Option { let metadata = tree .get_metadata(id) .expect("metadata missing for instance present in tree"); @@ -283,7 +296,7 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option< // path still exists. We can generate a snapshot starting at // that path and use it as the source for our patch. - let snapshot = match snapshot_from_vfs(&metadata.context, &vfs, &path) { + let snapshot = match snapshot_from_vfs(&metadata.context, &vfs, plugin_env, &path) { Ok(Some(snapshot)) => snapshot, Ok(None) => { log::error!( @@ -331,6 +344,7 @@ fn compute_and_apply_changes(tree: &mut RojoTree, vfs: &Vfs, id: Ref) -> Option< instance_name, project_node, &vfs, + plugin_env, parent_class.as_ref().map(|name| name.as_str()), ); diff --git a/src/lib.rs b/src/lib.rs index 51195b6c1..8c1291e21 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -10,10 +10,12 @@ mod tree_view; mod auth_cookie; mod change_processor; mod glob; +mod load_file; mod lua_ast; mod message_queue; mod multimap; mod path_serializer; +mod plugin_env; mod project; mod resolution; mod serve_session; diff --git a/src/load_file.rs b/src/load_file.rs new file mode 100644 index 000000000..53c2342a4 --- /dev/null +++ b/src/load_file.rs @@ -0,0 +1,20 @@ +use memofs::Vfs; +use std::{path::Path, sync::Arc}; + +use crate::plugin_env::PluginEnv; + +pub fn load_file( + vfs: &Vfs, + plugin_env: &PluginEnv, + path: &Path, +) -> Result>, anyhow::Error> { + let plugin_result = plugin_env.load(path.to_str().unwrap()); + match plugin_result { + Ok(Some(data)) => return Ok(Arc::new(data.as_bytes().to_vec())), + Ok(None) => {} + Err(_) => {} + } + + let contents = vfs.read(path)?; + return Ok(contents); +} diff --git a/src/lua_ast.rs b/src/lua_ast.rs index c31b609f9..088f9ed31 100644 --- a/src/lua_ast.rs +++ b/src/lua_ast.rs @@ -98,6 +98,13 @@ impl FmtLua for Expression { } } +impl fmt::Display for Expression { + fn fmt(&self, output: &mut fmt::Formatter) -> fmt::Result { + let mut stream = LuaStream::new(output); + FmtLua::fmt_lua(self, &mut stream) + } +} + impl From for Expression { fn from(value: String) -> Self { Self::String(value) diff --git a/src/plugin_env.rs b/src/plugin_env.rs new file mode 100644 index 000000000..18f1b6656 --- /dev/null +++ b/src/plugin_env.rs @@ -0,0 +1,183 @@ +use memofs::Vfs; +use rlua::{Function, Lua, StdLib, Table}; +use std::{fs, path::Path, str, str::FromStr, sync::Arc}; + +use crate::snapshot_middleware::SnapshotMiddleware; + +pub struct PluginEnv { + lua: Lua, + vfs: Arc, +} + +impl PluginEnv { + pub fn new(vfs: Arc) -> Self { + let lua = Lua::new_with( + StdLib::BASE + | StdLib::TABLE + | StdLib::STRING + | StdLib::UTF8 + | StdLib::MATH + | StdLib::PACKAGE, + ); + PluginEnv { lua, vfs } + } + + pub fn init(&self) -> Result<(), rlua::Error> { + self.lua.context(|lua_ctx| { + let globals = lua_ctx.globals(); + + let plugins_table = lua_ctx.create_table()?; + globals.set("plugins", plugins_table)?; + + let plugin_library_table = lua_ctx.create_table()?; + globals.set("rojo", plugin_library_table)?; + + Ok::<(), rlua::Error>(()) + }) + } + + pub fn context_with_vfs(&self, f: F) -> Result + where + F: FnOnce(rlua::Context) -> Result, + { + // We cannot just create a global function that has access to the vfs and call it whenever + // we want because that would be unsafe as Lua is unaware of the lifetime of vfs. Therefore + // we have to create a limited lifetime scope that has access to the vfs and define the + // function each time plugin code is executed. + let vfs = Arc::clone(&self.vfs); + + self.lua.context(|lua_ctx| { + lua_ctx.scope(|scope| { + let globals = lua_ctx.globals(); + let plugin_library_table: Table = globals.get("rojo")?; + let read_file_fn = scope.create_function_mut(|_, id: String| { + let path = Path::new(&id); + let contents = vfs.read(path).unwrap(); + let contents_str = str::from_utf8(&contents).unwrap(); + Ok::(contents_str.to_owned()) + })?; + plugin_library_table.set("readFileAsUtf8", read_file_fn)?; + + f(lua_ctx) + }) + }) + } + + fn load_plugin_source(&self, plugin_source: &str) -> String { + // TODO: Support downloading and caching plugins + fs::read_to_string(plugin_source).unwrap() + } + + pub fn load_plugin( + &self, + plugin_source: &str, + plugin_options: String, + ) -> Result<(), rlua::Error> { + let plugin_lua = &(self.load_plugin_source(plugin_source)); + + self.context_with_vfs(|lua_ctx| { + let globals = lua_ctx.globals(); + + let create_plugin_fn: Option = + lua_ctx.load(plugin_lua).set_name(plugin_source)?.eval()?; + let create_plugin_fn = match create_plugin_fn { + Some(v) => v, + None => { + return Err(rlua::Error::RuntimeError( + format!( + "plugin from source '{}' did not return a creation function.", + plugin_source + ) + .to_string(), + )) + } + }; + + let plugin_options_table: Table = lua_ctx + .load(&plugin_options) + .set_name("plugin options")? + .eval()?; + + let plugin_instance: Option = create_plugin_fn.call(plugin_options_table)?; + let plugin_instance = match plugin_instance { + Some(v) => v, + None => { + return Err(rlua::Error::RuntimeError( + format!( + "creation function for plugin from source '{}' did not return a plugin instance.", + plugin_source + ) + .to_string(), + )) + } + }; + + let plugin_name: Option = plugin_instance.get("name")?; + let plugin_name = match plugin_name.unwrap_or("".to_owned()) { + v if v.trim().is_empty() => { + return Err(rlua::Error::RuntimeError( + format!( + "plugin instance for plugin from source '{}' did not have a name.", + plugin_source + ) + .to_string(), + )) + } + v => v, + }; + + log::trace!( + "Loaded plugin '{}' from source: {}", + plugin_name, + plugin_source + ); + + let plugins_table: Table = globals.get("plugins")?; + plugins_table.set(plugins_table.len()? + 1, plugin_instance)?; + + Ok::<(), rlua::Error>(()) + }) + } + + pub fn middleware( + &self, + id: &str, + ) -> Result<(Option, Option), rlua::Error> { + self.context_with_vfs(|lua_ctx| { + let globals = lua_ctx.globals(); + + let plugins: Table = globals.get("plugins")?; + for plugin in plugins.sequence_values::
() { + let middleware_fn: Function = plugin?.get("middleware")?; + let (middleware_str, name): (Option, Option) = + middleware_fn.call(id)?; + let middleware_enum = match middleware_str { + Some(str) => SnapshotMiddleware::from_str(&str).ok(), + None => None, + }; + if middleware_enum.is_some() { + return Ok((middleware_enum, name)); + } + } + + Ok((None, None)) + }) + } + + pub fn load(&self, id: &str) -> Result, rlua::Error> { + self.context_with_vfs(|lua_ctx| { + let globals = lua_ctx.globals(); + + let plugins: Table = globals.get("plugins")?; + for plugin in plugins.sequence_values::
() { + let load_fn: Function = plugin?.get("load")?; + let load_str: Option = load_fn.call(id)?; + if load_str.is_some() { + return Ok(load_str); + } + } + + Ok(None) + }) + } +} diff --git a/src/project.rs b/src/project.rs index 088e6d976..dfb8857fb 100644 --- a/src/project.rs +++ b/src/project.rs @@ -31,6 +31,16 @@ enum Error { }, } +#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)] +#[serde(untagged)] +pub enum PluginDescription { + Source(String), + SourceWithOptions { + source: String, + options: serde_json::Value, + }, +} + /// Contains all of the configuration for a Rojo-managed project. /// /// Project files are stored in `.project.json` files. @@ -40,6 +50,9 @@ pub struct Project { /// The name of the top-level instance described by the project. pub name: String, + #[serde(default = "Vec::new", skip_serializing_if = "Vec::is_empty")] + pub plugins: Vec, + /// The tree of instances described by this project. Projects always /// describe at least one instance. pub tree: ProjectNode, diff --git a/src/serve_session.rs b/src/serve_session.rs index 13b8037cc..ff5239d5d 100644 --- a/src/serve_session.rs +++ b/src/serve_session.rs @@ -14,8 +14,10 @@ use thiserror::Error; use crate::{ change_processor::ChangeProcessor, + lua_ast::Expression, message_queue::MessageQueue, - project::{Project, ProjectError}, + plugin_env::PluginEnv, + project::{PluginDescription, Project, ProjectError}, session_id::SessionId, snapshot::{ apply_patch_set, compute_patch_set, AppliedPatchSet, InstanceContext, InstanceSnapshot, @@ -24,6 +26,27 @@ use crate::{ snapshot_middleware::snapshot_from_vfs, }; +// TODO: Centralize this (copied from json middleware) +fn json_to_lua_value(value: serde_json::Value) -> Expression { + use serde_json::Value; + + match value { + Value::Null => Expression::Nil, + Value::Bool(value) => Expression::Bool(value), + Value::Number(value) => Expression::Number(value.as_f64().unwrap()), + Value::String(value) => Expression::String(value), + Value::Array(values) => { + Expression::Array(values.into_iter().map(json_to_lua_value).collect()) + } + Value::Object(values) => Expression::table( + values + .into_iter() + .map(|(key, value)| (key.into(), json_to_lua_value(value))) + .collect(), + ), + } +} + /// Contains all of the state for a Rojo serve session. A serve session is used /// when we need to build a Rojo tree and possibly rebuild it when input files /// change. @@ -119,6 +142,32 @@ impl ServeSession { } }; + let vfs = Arc::new(vfs); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + match plugin_env.init() { + Ok(_) => (), + Err(e) => return Err(ServeSessionError::Plugin { source: e }), + }; + + for plugin_description in root_project.plugins.iter() { + let default_options = "{}".to_string(); + let (plugin_source, plugin_options) = match plugin_description { + PluginDescription::Source(source) => (source, default_options), + PluginDescription::SourceWithOptions { source, options } => { + (source, json_to_lua_value(options.to_owned()).to_string()) + } + }; + + let temp = project_path.with_file_name(plugin_source); + let plugin_source_path = temp.to_str().unwrap(); + + match plugin_env.load_plugin(plugin_source_path, plugin_options) { + Ok(_) => (), + Err(e) => return Err(ServeSessionError::Plugin { source: e }), + }; + } + let mut tree = RojoTree::new(InstanceSnapshot::new()); let root_id = tree.get_root_id(); @@ -126,7 +175,7 @@ impl ServeSession { let instance_context = InstanceContext::default(); log::trace!("Generating snapshot of instances from VFS"); - let snapshot = snapshot_from_vfs(&instance_context, &vfs, &start_path)? + let snapshot = snapshot_from_vfs(&instance_context, &vfs, &plugin_env, &start_path)? .expect("snapshot did not return an instance"); log::trace!("Computing initial patch set"); @@ -140,7 +189,7 @@ impl ServeSession { let tree = Arc::new(Mutex::new(tree)); let message_queue = Arc::new(message_queue); - let vfs = Arc::new(vfs); + let plugin_env = Arc::new(Mutex::new(plugin_env)); let (tree_mutation_sender, tree_mutation_receiver) = crossbeam_channel::unbounded(); @@ -148,6 +197,7 @@ impl ServeSession { let change_processor = ChangeProcessor::start( Arc::clone(&tree), Arc::clone(&vfs), + Arc::clone(&plugin_env), Arc::clone(&message_queue), tree_mutation_receiver, ); @@ -240,4 +290,10 @@ pub enum ServeSessionError { #[from] source: anyhow::Error, }, + + #[error(transparent)] + Plugin { + #[from] + source: rlua::Error, + }, } diff --git a/src/snapshot_middleware/csv.rs b/src/snapshot_middleware/csv.rs index a8f271808..b6e282897 100644 --- a/src/snapshot_middleware/csv.rs +++ b/src/snapshot_middleware/csv.rs @@ -5,19 +5,24 @@ use maplit::hashmap; use memofs::{IoResultExt, Vfs}; use serde::Serialize; -use crate::snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}; +use crate::{ + load_file::load_file, + plugin_env::PluginEnv, + snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}, +}; -use super::{meta_file::AdjacentMetadata, util::PathExt}; +use super::meta_file::AdjacentMetadata; pub fn snapshot_csv( _context: &InstanceContext, vfs: &Vfs, + plugin_env: &PluginEnv, path: &Path, + name: &str, ) -> anyhow::Result> { - let name = path.file_name_trim_end(".csv")?; - + // TODO: This is probably broken + let contents = load_file(vfs, plugin_env, path)?; let meta_path = path.with_file_name(format!("{}.meta.json", name)); - let contents = vfs.read(path)?; let table_contents = convert_localization_csv(&contents).with_context(|| { format!( @@ -125,6 +130,8 @@ fn convert_localization_csv(contents: &[u8]) -> Result { #[cfg(test)] mod test { + use std::sync::Arc; + use super::*; use memofs::{InMemoryFs, VfsSnapshot}; @@ -142,12 +149,20 @@ Ack,Ack!,,An exclamation of despair,¡Ay!"#, ) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); - let instance_snapshot = - snapshot_csv(&InstanceContext::default(), &mut vfs, Path::new("/foo.csv")) - .unwrap() - .unwrap(); + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); + + let instance_snapshot = snapshot_csv( + &InstanceContext::default(), + &mut vfs, + &plugin_env, + Path::new("/foo.csv"), + "foo", + ) + .unwrap() + .unwrap(); insta::assert_yaml_snapshot!(instance_snapshot); } @@ -170,12 +185,20 @@ Ack,Ack!,,An exclamation of despair,¡Ay!"#, ) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); - let instance_snapshot = - snapshot_csv(&InstanceContext::default(), &mut vfs, Path::new("/foo.csv")) - .unwrap() - .unwrap(); + let instance_snapshot = snapshot_csv( + &InstanceContext::default(), + &mut vfs, + &plugin_env, + Path::new("/foo.csv"), + "foo", + ) + .unwrap() + .unwrap(); insta::assert_yaml_snapshot!(instance_snapshot); } diff --git a/src/snapshot_middleware/dir.rs b/src/snapshot_middleware/dir.rs index bad35c813..f85560d7e 100644 --- a/src/snapshot_middleware/dir.rs +++ b/src/snapshot_middleware/dir.rs @@ -2,13 +2,17 @@ use std::path::Path; use memofs::{DirEntry, IoResultExt, Vfs}; -use crate::snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}; +use crate::{ + plugin_env::PluginEnv, + snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}, +}; use super::{meta_file::DirectoryMetadata, snapshot_from_vfs}; pub fn snapshot_dir( context: &InstanceContext, vfs: &Vfs, + plugin_env: &PluginEnv, path: &Path, ) -> anyhow::Result> { let passes_filter_rules = |child: &DirEntry| { @@ -27,7 +31,7 @@ pub fn snapshot_dir( continue; } - if let Some(child_snapshot) = snapshot_from_vfs(context, vfs, entry.path())? { + if let Some(child_snapshot) = snapshot_from_vfs(context, vfs, plugin_env, entry.path())? { snapshot_children.push(child_snapshot); } } @@ -73,6 +77,8 @@ pub fn snapshot_dir( #[cfg(test)] mod test { + use std::sync::Arc; + use super::*; use maplit::hashmap; @@ -84,12 +90,19 @@ mod test { imfs.load_snapshot("/foo", VfsSnapshot::empty_dir()) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); - let instance_snapshot = - snapshot_dir(&InstanceContext::default(), &mut vfs, Path::new("/foo")) - .unwrap() - .unwrap(); + let instance_snapshot = snapshot_dir( + &InstanceContext::default(), + &mut vfs, + &plugin_env, + Path::new("/foo"), + ) + .unwrap() + .unwrap(); insta::assert_yaml_snapshot!(instance_snapshot); } @@ -105,12 +118,19 @@ mod test { ) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); - let instance_snapshot = - snapshot_dir(&InstanceContext::default(), &mut vfs, Path::new("/foo")) - .unwrap() - .unwrap(); + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); + + let instance_snapshot = snapshot_dir( + &InstanceContext::default(), + &mut vfs, + &plugin_env, + Path::new("/foo"), + ) + .unwrap() + .unwrap(); insta::assert_yaml_snapshot!(instance_snapshot); } diff --git a/src/snapshot_middleware/json.rs b/src/snapshot_middleware/json.rs index 8c7f369e3..7e47cf208 100644 --- a/src/snapshot_middleware/json.rs +++ b/src/snapshot_middleware/json.rs @@ -5,19 +5,22 @@ use maplit::hashmap; use memofs::{IoResultExt, Vfs}; use crate::{ + load_file::load_file, lua_ast::{Expression, Statement}, + plugin_env::PluginEnv, snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}, }; -use super::{meta_file::AdjacentMetadata, util::PathExt}; +use super::meta_file::AdjacentMetadata; pub fn snapshot_json( context: &InstanceContext, vfs: &Vfs, + plugin_env: &PluginEnv, path: &Path, + name: &str, ) -> anyhow::Result> { - let name = path.file_name_trim_end(".json")?; - let contents = vfs.read(path)?; + let contents = load_file(vfs, plugin_env, path)?; let value: serde_json::Value = serde_json::from_slice(&contents) .with_context(|| format!("File contains malformed JSON: {}", path.display()))?; @@ -75,6 +78,8 @@ fn json_to_lua_value(value: serde_json::Value) -> Expression { #[cfg(test)] mod test { + use std::sync::Arc; + use super::*; use memofs::{InMemoryFs, VfsSnapshot}; @@ -101,12 +106,17 @@ mod test { ) .unwrap(); - let mut vfs = Vfs::new(imfs.clone()); + let mut vfs = Arc::new(Vfs::new(imfs.clone())); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); let instance_snapshot = snapshot_json( &InstanceContext::default(), &mut vfs, + &plugin_env, Path::new("/foo.json"), + "foo", ) .unwrap() .unwrap(); diff --git a/src/snapshot_middleware/json_model.rs b/src/snapshot_middleware/json_model.rs index 520710312..d8694c9b6 100644 --- a/src/snapshot_middleware/json_model.rs +++ b/src/snapshot_middleware/json_model.rs @@ -5,20 +5,20 @@ use memofs::Vfs; use serde::Deserialize; use crate::{ + load_file::load_file, + plugin_env::PluginEnv, resolution::UnresolvedValue, snapshot::{InstanceContext, InstanceSnapshot}, }; -use super::util::PathExt; - pub fn snapshot_json_model( context: &InstanceContext, vfs: &Vfs, + plugin_env: &PluginEnv, path: &Path, + name: &str, ) -> anyhow::Result> { - let name = path.file_name_trim_end(".model.json")?; - - let contents = vfs.read(path)?; + let contents = load_file(vfs, plugin_env, path)?; let contents_str = str::from_utf8(&contents) .with_context(|| format!("File was not valid UTF-8: {}", path.display()))?; @@ -101,6 +101,8 @@ impl JsonModelCore { #[cfg(test)] mod test { + use std::sync::Arc; + use super::*; use memofs::{InMemoryFs, VfsSnapshot}; @@ -130,12 +132,17 @@ mod test { ) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); let instance_snapshot = snapshot_json_model( &InstanceContext::default(), &mut vfs, + &plugin_env, Path::new("/foo.model.json"), + "foo", ) .unwrap() .unwrap(); diff --git a/src/snapshot_middleware/lua.rs b/src/snapshot_middleware/lua.rs index d11e72c62..ef48adb1f 100644 --- a/src/snapshot_middleware/lua.rs +++ b/src/snapshot_middleware/lua.rs @@ -4,38 +4,33 @@ use anyhow::Context; use maplit::hashmap; use memofs::{IoResultExt, Vfs}; -use crate::snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}; +use crate::{ + load_file::load_file, + plugin_env::PluginEnv, + snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}, +}; -use super::{dir::snapshot_dir, meta_file::AdjacentMetadata, util::match_trailing}; +use super::{dir::snapshot_dir, meta_file::AdjacentMetadata}; /// Core routine for turning Lua files into snapshots. pub fn snapshot_lua( context: &InstanceContext, vfs: &Vfs, + plugin_env: &PluginEnv, path: &Path, + name: &str, + class_name: &str, ) -> anyhow::Result> { - let file_name = path.file_name().unwrap().to_string_lossy(); - - let (class_name, instance_name) = if let Some(name) = match_trailing(&file_name, ".server.lua") - { - ("Script", name) - } else if let Some(name) = match_trailing(&file_name, ".client.lua") { - ("LocalScript", name) - } else if let Some(name) = match_trailing(&file_name, ".lua") { - ("ModuleScript", name) - } else { - return Ok(None); - }; - - let contents = vfs.read(path)?; + let contents = load_file(vfs, plugin_env, path)?; let contents_str = str::from_utf8(&contents) .with_context(|| format!("File was not valid UTF-8: {}", path.display()))? .to_owned(); - let meta_path = path.with_file_name(format!("{}.meta.json", instance_name)); + // TODO: I think this is broken + let meta_path = path.with_file_name(format!("{}.meta.json", name)); let mut snapshot = InstanceSnapshot::new() - .name(instance_name) + .name(name) .class_name(class_name) .properties(hashmap! { "Source".to_owned() => contents_str.into(), @@ -63,10 +58,13 @@ pub fn snapshot_lua( pub fn snapshot_lua_init( context: &InstanceContext, vfs: &Vfs, + plugin_env: &PluginEnv, init_path: &Path, + name: &str, + class_name: &str, ) -> anyhow::Result> { let folder_path = init_path.parent().unwrap(); - let dir_snapshot = snapshot_dir(context, vfs, folder_path)?.unwrap(); + let dir_snapshot = snapshot_dir(context, vfs, plugin_env, folder_path)?.unwrap(); if dir_snapshot.class_name != "Folder" { anyhow::bail!( @@ -80,9 +78,10 @@ pub fn snapshot_lua_init( ); } - let mut init_snapshot = snapshot_lua(context, vfs, init_path)?.unwrap(); + let mut init_snapshot = + snapshot_lua(context, vfs, plugin_env, init_path, name, class_name)?.unwrap(); - init_snapshot.name = dir_snapshot.name; + // init_snapshot.name = dir_snapshot.name; init_snapshot.children = dir_snapshot.children; init_snapshot.metadata = dir_snapshot.metadata; @@ -91,6 +90,8 @@ pub fn snapshot_lua_init( #[cfg(test)] mod test { + use std::sync::Arc; + use super::*; use memofs::{InMemoryFs, VfsSnapshot}; @@ -101,12 +102,21 @@ mod test { imfs.load_snapshot("/foo.lua", VfsSnapshot::file("Hello there!")) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); - let instance_snapshot = - snapshot_lua(&InstanceContext::default(), &mut vfs, Path::new("/foo.lua")) - .unwrap() - .unwrap(); + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); + + let instance_snapshot = snapshot_lua( + &InstanceContext::default(), + &mut vfs, + &plugin_env, + Path::new("/foo.lua"), + "foo", + "ModuleScript", + ) + .unwrap() + .unwrap(); insta::assert_yaml_snapshot!(instance_snapshot); } @@ -117,12 +127,18 @@ mod test { imfs.load_snapshot("/foo.server.lua", VfsSnapshot::file("Hello there!")) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); let instance_snapshot = snapshot_lua( &InstanceContext::default(), &mut vfs, + &plugin_env, Path::new("/foo.server.lua"), + "foo", + "Script", ) .unwrap() .unwrap(); @@ -136,12 +152,18 @@ mod test { imfs.load_snapshot("/foo.client.lua", VfsSnapshot::file("Hello there!")) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); let instance_snapshot = snapshot_lua( &InstanceContext::default(), &mut vfs, + &plugin_env, Path::new("/foo.client.lua"), + "foo", + "LocalScript", ) .unwrap() .unwrap(); @@ -161,12 +183,21 @@ mod test { ) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); - let instance_snapshot = - snapshot_lua(&InstanceContext::default(), &mut vfs, Path::new("/root")) - .unwrap() - .unwrap(); + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); + + let instance_snapshot = snapshot_lua( + &InstanceContext::default(), + &mut vfs, + &plugin_env, + Path::new("/root"), + "root", + "ModuleScript", + ) + .unwrap() + .unwrap(); insta::assert_yaml_snapshot!(instance_snapshot); } @@ -188,12 +219,21 @@ mod test { ) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); - let instance_snapshot = - snapshot_lua(&InstanceContext::default(), &mut vfs, Path::new("/foo.lua")) - .unwrap() - .unwrap(); + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); + + let instance_snapshot = snapshot_lua( + &InstanceContext::default(), + &mut vfs, + &plugin_env, + Path::new("/foo.lua"), + "foo", + "ModuleScript", + ) + .unwrap() + .unwrap(); insta::assert_yaml_snapshot!(instance_snapshot); } @@ -215,12 +255,18 @@ mod test { ) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); let instance_snapshot = snapshot_lua( &InstanceContext::default(), &mut vfs, + &plugin_env, Path::new("/foo.server.lua"), + "foo", + "Script", ) .unwrap() .unwrap(); @@ -247,12 +293,18 @@ mod test { ) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); let instance_snapshot = snapshot_lua( &InstanceContext::default(), &mut vfs, + &plugin_env, Path::new("/bar.server.lua"), + "bar", + "Script", ) .unwrap() .unwrap(); diff --git a/src/snapshot_middleware/mod.rs b/src/snapshot_middleware/mod.rs index 36729f8f3..afff70750 100644 --- a/src/snapshot_middleware/mod.rs +++ b/src/snapshot_middleware/mod.rs @@ -17,11 +17,14 @@ mod rbxmx; mod txt; mod util; -use std::path::Path; +use std::{path::Path, str::FromStr}; use memofs::{IoResultExt, Vfs}; -use crate::snapshot::{InstanceContext, InstanceSnapshot}; +use crate::{ + plugin_env::PluginEnv, + snapshot::{InstanceContext, InstanceSnapshot}, +}; use self::{ csv::snapshot_csv, @@ -38,11 +41,48 @@ use self::{ pub use self::project::snapshot_project_node; +#[derive(Debug)] +pub enum SnapshotMiddleware { + Csv, + Dir, + Json, + JsonModel, + LuaModule, + LuaClient, + LuaServer, + Project, + Rbxm, + Rbxmx, + Txt, +} + +impl FromStr for SnapshotMiddleware { + type Err = String; + + fn from_str(s: &str) -> Result { + match s { + "csv" => Ok(SnapshotMiddleware::Csv), + "dir" => Ok(SnapshotMiddleware::Dir), + "json" => Ok(SnapshotMiddleware::Json), + "json_model" => Ok(SnapshotMiddleware::JsonModel), + "lua_module" => Ok(SnapshotMiddleware::LuaModule), + "lua_server" => Ok(SnapshotMiddleware::LuaServer), + "lua_client" => Ok(SnapshotMiddleware::LuaClient), + "project" => Ok(SnapshotMiddleware::Project), + "rbxm" => Ok(SnapshotMiddleware::Rbxm), + "rbxmx" => Ok(SnapshotMiddleware::Rbxmx), + "txt" => Ok(SnapshotMiddleware::Txt), + _ => Err(format!("Unknown snapshot middleware: {}", s)), + } + } +} + /// The main entrypoint to the snapshot function. This function can be pointed /// at any path and will return something if Rojo knows how to deal with it. pub fn snapshot_from_vfs( context: &InstanceContext, vfs: &Vfs, + plugin_env: &PluginEnv, path: &Path, ) -> anyhow::Result> { let meta = match vfs.metadata(path).with_not_found()? { @@ -50,56 +90,167 @@ pub fn snapshot_from_vfs( None => return Ok(None), }; + // TODO: Think about how to handle this stuff for plugins. if meta.is_dir() { let project_path = path.join("default.project.json"); if vfs.metadata(&project_path).with_not_found()?.is_some() { - return snapshot_project(context, vfs, &project_path); + return snapshot_project(context, vfs, plugin_env, &project_path); } let init_path = path.join("init.lua"); if vfs.metadata(&init_path).with_not_found()?.is_some() { - return snapshot_lua_init(context, vfs, &init_path); + return snapshot_lua_init( + context, + vfs, + plugin_env, + &init_path, + &path.file_name().unwrap().to_string_lossy(), + "ModuleScript", + ); } let init_path = path.join("init.server.lua"); if vfs.metadata(&init_path).with_not_found()?.is_some() { - return snapshot_lua_init(context, vfs, &init_path); + return snapshot_lua_init( + context, + vfs, + plugin_env, + &init_path, + &path.file_name().unwrap().to_string_lossy(), + "Script", + ); } let init_path = path.join("init.client.lua"); if vfs.metadata(&init_path).with_not_found()?.is_some() { - return snapshot_lua_init(context, vfs, &init_path); + return snapshot_lua_init( + context, + vfs, + plugin_env, + &init_path, + &path.file_name().unwrap().to_string_lossy(), + "LocalScript", + ); } - snapshot_dir(context, vfs, path) + snapshot_dir(context, vfs, plugin_env, path) } else { - if let Ok(name) = path.file_name_trim_end(".lua") { - match name { - // init scripts are handled elsewhere and should not turn into - // their own children. - "init" | "init.client" | "init.server" => return Ok(None), - - _ => return snapshot_lua(context, vfs, path), - } - } else if path.file_name_ends_with(".project.json") { - return snapshot_project(context, vfs, path); - } else if path.file_name_ends_with(".model.json") { - return snapshot_json_model(context, vfs, path); - } else if path.file_name_ends_with(".meta.json") { - // .meta.json files do not turn into their own instances. - return Ok(None); - } else if path.file_name_ends_with(".json") { - return snapshot_json(context, vfs, path); - } else if path.file_name_ends_with(".csv") { - return snapshot_csv(context, vfs, path); - } else if path.file_name_ends_with(".txt") { - return snapshot_txt(context, vfs, path); - } else if path.file_name_ends_with(".rbxmx") { - return snapshot_rbxmx(context, vfs, path); - } else if path.file_name_ends_with(".rbxm") { - return snapshot_rbxm(context, vfs, path); + let mut middleware: (Option, Option) = + plugin_env.middleware(path.to_str().unwrap())?; + + if !matches!(middleware, (Some(_), _)) { + middleware = if let Ok(name) = path.file_name_trim_end(".lua") { + match name { + "init" | "init.client" | "init.server" => (None, None), + _ => { + if let Ok(name) = path.file_name_trim_end(".server.lua") { + (Some(SnapshotMiddleware::LuaServer), Some(name.to_owned())) + } else if let Ok(name) = path.file_name_trim_end(".client.lua") { + (Some(SnapshotMiddleware::LuaClient), Some(name.to_owned())) + } else { + (Some(SnapshotMiddleware::LuaModule), Some(name.to_owned())) + } + } + } + } else if path.file_name_ends_with(".project.json") { + ( + Some(SnapshotMiddleware::Project), + match path.file_name_trim_end(".project.json") { + Ok(v) => Some(v.to_owned()), + Err(_) => None, + }, + ) + } else if path.file_name_ends_with(".model.json") { + ( + Some(SnapshotMiddleware::JsonModel), + match path.file_name_trim_end(".model.json") { + Ok(v) => Some(v.to_owned()), + Err(_) => None, + }, + ) + } else if path.file_name_ends_with(".meta.json") { + // .meta.json files do not turn into their own instances. + (None, None) + } else if path.file_name_ends_with(".json") { + ( + Some(SnapshotMiddleware::Json), + match path.file_name_trim_end(".json") { + Ok(v) => Some(v.to_owned()), + Err(_) => None, + }, + ) + } else if path.file_name_ends_with(".csv") { + ( + Some(SnapshotMiddleware::Csv), + match path.file_name_trim_end(".csv") { + Ok(v) => Some(v.to_owned()), + Err(_) => None, + }, + ) + } else if path.file_name_ends_with(".txt") { + ( + Some(SnapshotMiddleware::Txt), + match path.file_name_trim_end(".txt") { + Ok(v) => Some(v.to_owned()), + Err(_) => None, + }, + ) + } else if path.file_name_ends_with(".rbxmx") { + ( + Some(SnapshotMiddleware::Rbxmx), + match path.file_name_trim_end(".rbxmx") { + Ok(v) => Some(v.to_owned()), + Err(_) => None, + }, + ) + } else if path.file_name_ends_with(".rbxm") { + ( + Some(SnapshotMiddleware::Rbxm), + match path.file_name_trim_end(".rbxm") { + Ok(v) => Some(v.to_owned()), + Err(_) => None, + }, + ) + } else { + (None, None) + }; } - Ok(None) + middleware = match middleware { + // Pick a default name (name without extension) + (Some(x), None) => ( + Some(x), + match path.file_name_no_extension() { + Ok(v) => Some(v.to_owned()), + Err(_) => None, + }, + ), + x => x, + }; + + return match middleware { + (Some(x), Some(name)) => match x { + SnapshotMiddleware::LuaModule => { + snapshot_lua(context, vfs, &plugin_env, path, &name, "ModuleScript") + } + SnapshotMiddleware::LuaServer => { + snapshot_lua(context, vfs, &plugin_env, path, &name, "Script") + } + SnapshotMiddleware::LuaClient => { + snapshot_lua(context, vfs, &plugin_env, path, &name, "LocalScript") + } + SnapshotMiddleware::Project => snapshot_project(context, vfs, plugin_env, path), + SnapshotMiddleware::JsonModel => { + snapshot_json_model(context, vfs, plugin_env, path, &name) + } + SnapshotMiddleware::Json => snapshot_json(context, vfs, plugin_env, path, &name), + SnapshotMiddleware::Csv => snapshot_csv(context, vfs, plugin_env, path, &name), + SnapshotMiddleware::Txt => snapshot_txt(context, vfs, plugin_env, path, &name), + SnapshotMiddleware::Rbxmx => snapshot_rbxmx(context, vfs, plugin_env, path, &name), + SnapshotMiddleware::Rbxm => snapshot_rbxm(context, vfs, plugin_env, path, &name), + _ => Ok(None), + }, + _ => Ok(None), + }; } } diff --git a/src/snapshot_middleware/project.rs b/src/snapshot_middleware/project.rs index 673def0fd..fd6f6905f 100644 --- a/src/snapshot_middleware/project.rs +++ b/src/snapshot_middleware/project.rs @@ -5,6 +5,8 @@ use memofs::Vfs; use rbx_reflection::ClassTag; use crate::{ + load_file::load_file, + plugin_env::PluginEnv, project::{Project, ProjectNode}, snapshot::{ InstanceContext, InstanceMetadata, InstanceSnapshot, InstigatingSource, PathIgnoreRule, @@ -16,9 +18,11 @@ use super::snapshot_from_vfs; pub fn snapshot_project( context: &InstanceContext, vfs: &Vfs, + plugin_env: &PluginEnv, path: &Path, ) -> anyhow::Result> { - let project = Project::load_from_slice(&vfs.read(path)?, path) + let contents = load_file(vfs, plugin_env, path)?; + let project = Project::load_from_slice(&contents, path) .with_context(|| format!("File was not a valid Rojo project: {}", path.display()))?; let mut context = context.clone(); @@ -32,8 +36,16 @@ pub fn snapshot_project( // TODO: If this project node is a path to an instance that Rojo doesn't // understand, this may panic! - let mut snapshot = - snapshot_project_node(&context, path, &project.name, &project.tree, vfs, None)?.unwrap(); + let mut snapshot = snapshot_project_node( + &context, + path, + &project.name, + &project.tree, + vfs, + plugin_env, + None, + )? + .unwrap(); // Setting the instigating source to the project file path is a little // coarse. @@ -62,6 +74,7 @@ pub fn snapshot_project_node( instance_name: &str, node: &ProjectNode, vfs: &Vfs, + plugin_env: &PluginEnv, parent_class: Option<&str>, ) -> anyhow::Result> { let project_folder = project_path.parent().unwrap(); @@ -86,7 +99,7 @@ pub fn snapshot_project_node( Cow::Borrowed(path) }; - if let Some(snapshot) = snapshot_from_vfs(context, vfs, &path)? { + if let Some(snapshot) = snapshot_from_vfs(context, vfs, plugin_env, &path)? { class_name_from_path = Some(snapshot.class_name); // Properties from the snapshot are pulled in unchanged, and @@ -182,6 +195,7 @@ pub fn snapshot_project_node( child_name, child_project_node, vfs, + plugin_env, Some(&class_name), )? { children.push(child); @@ -276,6 +290,8 @@ fn infer_class_name(name: &str, parent_class: Option<&str>) -> Option anyhow::Result> { - let name = path.file_name_trim_end(".rbxm")?; - - let temp_tree = rbx_binary::from_reader(vfs.read(path)?.as_slice()) + let contents = load_file(vfs, plugin_env, path)?; + let temp_tree = rbx_binary::from_reader(contents.as_slice()) .with_context(|| format!("Malformed rbxm file: {}", path.display()))?; let root_instance = temp_tree.root(); @@ -42,6 +45,8 @@ pub fn snapshot_rbxm( #[cfg(test)] mod test { + use std::sync::Arc; + use super::*; use memofs::{InMemoryFs, VfsSnapshot}; @@ -55,12 +60,17 @@ mod test { ) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); let instance_snapshot = snapshot_rbxm( &InstanceContext::default(), &mut vfs, + &plugin_env, Path::new("/foo.rbxm"), + "foo", ) .unwrap() .unwrap(); diff --git a/src/snapshot_middleware/rbxmx.rs b/src/snapshot_middleware/rbxmx.rs index 3cdd52e45..efe9b4835 100644 --- a/src/snapshot_middleware/rbxmx.rs +++ b/src/snapshot_middleware/rbxmx.rs @@ -3,21 +3,24 @@ use std::path::Path; use anyhow::Context; use memofs::Vfs; -use crate::snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}; - -use super::util::PathExt; +use crate::{ + load_file::load_file, + plugin_env::PluginEnv, + snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}, +}; pub fn snapshot_rbxmx( context: &InstanceContext, vfs: &Vfs, + plugin_env: &PluginEnv, path: &Path, + name: &str, ) -> anyhow::Result> { - let name = path.file_name_trim_end(".rbxmx")?; - let options = rbx_xml::DecodeOptions::new() .property_behavior(rbx_xml::DecodePropertyBehavior::ReadUnknown); - let temp_tree = rbx_xml::from_reader(vfs.read(path)?.as_slice(), options) + let contents = load_file(vfs, plugin_env, path)?; + let temp_tree = rbx_xml::from_reader(contents.as_slice(), options) .with_context(|| format!("Malformed rbxm file: {}", path.display()))?; let root_instance = temp_tree.root(); @@ -45,6 +48,8 @@ pub fn snapshot_rbxmx( #[cfg(test)] mod test { + use std::sync::Arc; + use super::*; use memofs::{InMemoryFs, VfsSnapshot}; @@ -68,12 +73,17 @@ mod test { ) .unwrap(); - let mut vfs = Vfs::new(imfs); + let mut vfs = Arc::new(Vfs::new(imfs)); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); let instance_snapshot = snapshot_rbxmx( &InstanceContext::default(), &mut vfs, + &plugin_env, Path::new("/foo.rbxmx"), + "foo", ) .unwrap() .unwrap(); diff --git a/src/snapshot_middleware/txt.rs b/src/snapshot_middleware/txt.rs index 13d5b9907..5cac2cb65 100644 --- a/src/snapshot_middleware/txt.rs +++ b/src/snapshot_middleware/txt.rs @@ -4,18 +4,22 @@ use anyhow::Context; use maplit::hashmap; use memofs::{IoResultExt, Vfs}; -use crate::snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}; +use crate::{ + load_file::load_file, + plugin_env::PluginEnv, + snapshot::{InstanceContext, InstanceMetadata, InstanceSnapshot}, +}; -use super::{meta_file::AdjacentMetadata, util::PathExt}; +use super::meta_file::AdjacentMetadata; pub fn snapshot_txt( context: &InstanceContext, vfs: &Vfs, + plugin_env: &PluginEnv, path: &Path, + name: &str, ) -> anyhow::Result> { - let name = path.file_name_trim_end(".txt")?; - - let contents = vfs.read(path)?; + let contents = load_file(vfs, plugin_env, path)?; let contents_str = str::from_utf8(&contents) .with_context(|| format!("File was not valid UTF-8: {}", path.display()))? .to_owned(); @@ -47,6 +51,8 @@ pub fn snapshot_txt( #[cfg(test)] mod test { + use std::sync::Arc; + use super::*; use memofs::{InMemoryFs, VfsSnapshot}; @@ -57,12 +63,20 @@ mod test { imfs.load_snapshot("/foo.txt", VfsSnapshot::file("Hello there!")) .unwrap(); - let mut vfs = Vfs::new(imfs.clone()); + let mut vfs = Arc::new(Vfs::new(imfs)); + + let plugin_env = PluginEnv::new(Arc::clone(&vfs)); + plugin_env.init().unwrap(); - let instance_snapshot = - snapshot_txt(&InstanceContext::default(), &mut vfs, Path::new("/foo.txt")) - .unwrap() - .unwrap(); + let instance_snapshot = snapshot_txt( + &InstanceContext::default(), + &mut vfs, + &plugin_env, + Path::new("/foo.txt"), + "foo", + ) + .unwrap() + .unwrap(); insta::assert_yaml_snapshot!(instance_snapshot); } diff --git a/src/snapshot_middleware/util.rs b/src/snapshot_middleware/util.rs index 517cd6066..6cf439226 100644 --- a/src/snapshot_middleware/util.rs +++ b/src/snapshot_middleware/util.rs @@ -1,4 +1,4 @@ -use std::path::Path; +use std::{ops::Index, path::Path}; use anyhow::Context; @@ -16,6 +16,7 @@ pub fn match_trailing<'a>(input: &'a str, suffix: &str) -> Option<&'a str> { pub trait PathExt { fn file_name_ends_with(&self, suffix: &str) -> bool; fn file_name_trim_end<'a>(&'a self, suffix: &str) -> anyhow::Result<&'a str>; + fn file_name_no_extension<'a>(&'a self) -> anyhow::Result<&'a str>; } impl

PathExt for P @@ -40,4 +41,15 @@ where match_trailing(&file_name, suffix) .with_context(|| format!("Path did not end in {}: {}", suffix, path.display())) } + + fn file_name_no_extension<'a>(&'a self) -> anyhow::Result<&'a str> { + let path = self.as_ref(); + let file_name = path + .file_name() + .and_then(|name| name.to_str()) + .with_context(|| format!("Path did not have a file name: {}", path.display()))?; + + let index = file_name.chars().position(|c| c == '.').unwrap_or(0); + Ok(&file_name[0..index]) + } } diff --git a/test-projects/plugins/default.project.json b/test-projects/plugins/default.project.json index 02387e7e3..61bdc59f3 100644 --- a/test-projects/plugins/default.project.json +++ b/test-projects/plugins/default.project.json @@ -1,9 +1,17 @@ { "name": "plugins", - "tree": { - "$path": "src" - }, "plugins": [ - "test-plugin.lua" - ] -} \ No newline at end of file + "fake-moonscript.lua", + { + "source": "load-as-stringvalue.lua", + "options": { "extensions": ["md"] } + } + ], + "tree": { + "$className": "DataModel", + "ReplicatedStorage": { + "$className": "ReplicatedStorage", + "$path": "src" + } + } +} diff --git a/test-projects/plugins/fake-moonscript.lua b/test-projects/plugins/fake-moonscript.lua new file mode 100644 index 000000000..5910875ed --- /dev/null +++ b/test-projects/plugins/fake-moonscript.lua @@ -0,0 +1,39 @@ +print('[plugin(fake-moonscript)] loading') + +-- This does not actually compile moonscript, it is just to test the hooks that would be used for a +-- real one. + +local function compile(moonscript) + return moonscript +end + +return function(options) + print('[plugin(fake-moonscript)] create') + + return { + name = 'fake-moonscript', + middleware = function(id) + print(('[plugin(fake-moonscript)] middleware: %s'):format(id)) + if id:match('%.moon$') then + print('[plugin(fake-moonscript)] matched') + if id:match('%.server%.moon$') then + return 'lua_server' + elseif id:match('%.client%.moon$') then + return 'lua_client' + else + return 'lua_module' + end + end + print('[plugin(fake-moonscript)] skipping') + end, + load = function(id) + print(('[plugin(fake-moonscript)] load: %s'):format(id)) + if id:match('%.moon$') then + print('[plugin(fake-moonscript)] matched') + local contents = rojo.readFileAsUtf8(id) + return compile(contents) + end + print('[plugin(fake-moonscript)] skipping') + end + } +end diff --git a/test-projects/plugins/load-as-stringvalue.lua b/test-projects/plugins/load-as-stringvalue.lua new file mode 100644 index 000000000..668429d26 --- /dev/null +++ b/test-projects/plugins/load-as-stringvalue.lua @@ -0,0 +1,54 @@ +print('[plugin(load-as-stringvalue)] loading') + +local function tableToString(t) + local s = '' + if type(t) == 'table' then + s = s .. '{ ' + for k, v in next, t do + if type(k) == 'number' then + s = s .. tableToString(v) + else + s = s .. k .. ' = ' .. tableToString(v) + end + end + s = s .. ' }' + elseif type(t) == 'string' then + s = s .. '"' .. t .. '"' + else + s = s .. tostring(t) + end + return s +end + +return function(options) + print(('[plugin(load-as-stringvalue)] create with: %s'):format(tableToString(options))) + options.extensions = options.extensions or {} + + return { + name = 'load-as-stringvalue', + middleware = function(id) + print(('[plugin(load-as-stringvalue)] middleware: %s'):format(id)) + local idExt = id:match('%.(%w+)$') + for _, ext in next, options.extensions do + if ext == idExt then + print(('[plugin(load-as-stringvalue)] matched: %s'):format(ext)) + return 'json_model' + end + end + print('[plugin(load-as-stringvalue)] skipping') + end, + load = function(id) + print(('[plugin(load-as-stringvalue)] load: %s'):format(id)) + local idExt = id:match('%.(%w+)$') + for _, ext in next, options.extensions do + if ext == idExt then + local contents = rojo.readFileAsUtf8(id) + print(('[plugin(load-as-stringvalue)] matched: %s'):format(ext)) + local encoded = contents:gsub('\n', '\\n') + return ('{"ClassName": "StringValue", "Properties": { "Value": "%s" }}'):format(encoded) + end + end + print('[plugin(load-as-stringvalue)] skipping') + end + } +end diff --git a/test-projects/plugins/src/Document.md b/test-projects/plugins/src/Document.md new file mode 100644 index 000000000..6e2652bd1 --- /dev/null +++ b/test-projects/plugins/src/Document.md @@ -0,0 +1,3 @@ +# Document + +A **bold** statement made in Markdown. diff --git a/test-projects/plugins/src/Hello.client.moon b/test-projects/plugins/src/Hello.client.moon new file mode 100644 index 000000000..f8fce2672 --- /dev/null +++ b/test-projects/plugins/src/Hello.client.moon @@ -0,0 +1 @@ +print "Hello from the client!" \ No newline at end of file diff --git a/test-projects/plugins/src/Hello.moon b/test-projects/plugins/src/Hello.moon new file mode 100644 index 000000000..057da09bb --- /dev/null +++ b/test-projects/plugins/src/Hello.moon @@ -0,0 +1 @@ +return "Hello from a module!" diff --git a/test-projects/plugins/src/Hello.server.moon b/test-projects/plugins/src/Hello.server.moon new file mode 100644 index 000000000..7526d7267 --- /dev/null +++ b/test-projects/plugins/src/Hello.server.moon @@ -0,0 +1 @@ +print "Hello from the server!" diff --git a/test-projects/plugins/src/hello.moon b/test-projects/plugins/src/hello.moon deleted file mode 100644 index ddb781cf6..000000000 --- a/test-projects/plugins/src/hello.moon +++ /dev/null @@ -1 +0,0 @@ -print 'Hello, world!' \ No newline at end of file diff --git a/test-projects/plugins/test-plugin.lua b/test-projects/plugins/test-plugin.lua deleted file mode 100644 index 147463cb2..000000000 --- a/test-projects/plugins/test-plugin.lua +++ /dev/null @@ -1,20 +0,0 @@ -print("test-plugin initializing...") - -return function(nextDispatch, entry) - if entry:isDirectory() then - return nextDispatch(entry) - end - - local name = entry:fileName() - local instanceName = name:match("(.-)%.moon$") - - if instanceName == nil then - return nextDispatch(entry) - end - - return rojo.instance({ - Name = instanceName, - ClassName = "ModuleScript", - Source = compileMoonScript(entry:contents()), - }) -end \ No newline at end of file