added script to upload vanilla .svelte components.

[BlackFenix2]

I added a script to preprocess svelte files with different Template,Script and Style languages and spit out vanilla .svelte files. This is to serve 2 use cases for a component library.

• Sapper-friendly compiled .svelte components
• Allows use by consumers that do not have preprocessors in their current project

This may help Sapper projects more than anything else, bare bones svelte projects shouldn't have a problem loading modules.

[dummdidumm]

Thanks for this! I added some comments in the code. Also, you need to remove your package-lock.json from the PR.

On a broader note: I thought about going the route of a script, too, but the problem is that other files might need preprocessing, too. If you use TypeScript inside your Svelte files, you will likely also write .ts files, not .js, and those need preprocessing, too. What to do about this? I'm not sure. Several options come to my mind:

• Assume that the only file type that needs preprocessing outside of Svelte files are TS files (likely the 80% use case) and do one of these three:
  • add code specific to transpiling ts code. But this would mean adding a TS dependency which will bloat the template even for everyone who is not using it.
  • add docs on how to tackle this for TS with some code snippets to get going fast.
  • add a script which will add the TS dependency and other changes required to get going with TS (similar to the script in the template-repo) (my favorite)
• Add an extension point to the script, maybe some kind of hook users can fill if they need to preprocess other files. But this will likely not work for TS because TS needs the whole picture, not transpilation on a file-per-file basis.

[dummdidumm]

Just copying other files will not work if they need preprocessing, too, like TS files.

[BlackFenix2]

The svelte pre-processor can handle the .svelte files, i was thinking of using the typescript plugin for rollup to compile the .ts files after the preprocessing. I can just get rid of the copy file logic to prevent the .ts files from going into the output folder.

as an alternative, i could change the postbuild to a prebuild script so that the preprocessor can spit our compiled .svelte files and then have rollup bundle/minify the remaining .svelte and .ts files.

[firefish5000]

Note typescript (and maybe other) compiler can simply be called directly on the source with the output directory specified.
tsc --declaration --outDir dist
It will then compile the files over to the out directory. Done this way, I would say copying the files over is unnecessary We still need to copy though so non-preprocessed files can be imported, like plain js. Perhaps we should only preprocess svelte files in this script, and let any non-svelte files be handled separately afterwards (possibly by their native compilers).

[firefish5000]

Failing to see this, I wrote one as well.
POC is available via npx @tdi/svcli --preprocess.
Note though the project is a badly and blindly written scaffold cli for my (eventual?) personal use that should not be relied on.
Rest of this post is in regards to this PR & script.

As far as non-svelte files are concerned. As much as I would like better integration, I believe typescript, postcss, and etc.
can use their own compilers as per norm without any issue. We could simply tell users to add lines to package.json like

package.json

...
"scripts": {
  "preprocess": "npm run preprocess:svelte; npm run preprocess:typescript",
  "preprocess:svelte": "node script.js",
  "preprocess:typescript": "tsc --declaration --outDir wherever/preprocessed/files/go"
}
...

and add calls to all preprocessors in package.json via (possibly utilizing npm-run-all).
If you do want an all in one solution, this may be the tipping point at which it becomes worth it to write a plugin for rollup/webpack to dump the processed files.

As for improvements, the only major thing I can think of is using an arrays instead of just autoPreprocess() and .svelte extension so it is more obvious to users how to port their rollup/webpack config should they use other preprocessors (like mdsvex). And putting the arrays somewhere accessible, like the top of the script, or in its own config file (so rollup and webpack can import it and everything stays in sync).

svelte-preprocess.config.js

const sveltePreprocess = require("svelte-preprocess")
const {mdsvex} = require("mdsvex")
const generatePreprocess = ({sourceMap}={})=>[
    sveltePreprocess({sourceMap: sourceMap ?? false}),
    mdsvex(),
  ]
module.exports={
  generatePreprocess: generatePreprocess,
  preprocess: generatePreprocess(),
  extensions: [".svelte",".svx"],
  outDir: "./preprocessed",
  srcDir: "./src",
}

You can then use them in rollup, webpack, and the script. For the script, I would do

script.js (or a more descriptive filename)

   const {extensions,preprocessors} =  require('./svelte-preprocess.config.js')
   ...
   // process .svelte file
   // Use endsWith instead of includes so we don't process .svelte.disabled, .svelte.css and other improper extensions.
   if (extensions.some(ext=>file.endsWith(ext))) {
     // run autopreprocessor
     await parseSvelte(sourceFile, distFile);
   }
   ...
   svelte.preprocess(source, preprocessors, {
      filename: path.basename(destination),
    })

In my version, I also gave the ability to required setting the source directory and output directory. Not sure I would call that an improvement or addition of entirely pointless config options.

Dependencies

IMHO, There is no need real need for component-template to ship with svelte-preprocess, typescript, or any other preprocessors added to devDependencies.
However, svelte doesn't offers an easy way to output the preprocessed files. So we should at least ship with
this script, directions on how to use it, and a list of anything extra that must be installed to run it (should be none, as svelte-preprocessor is just one svelte processor implementation, not the all-in-one solution).

Actually, might be better if this script lived in its own package and we just had "preprocess": "npx preprocess-svelte" in package.json script, leaving installation entirely optional.

Also include some (ideally short) directions on how to enable compiling a few other common languages. I gave an example for typescript using tsc. If that is method is acceptable, just a short explanation of what it does should suffice.

[bluwy]

I also did something similar for my library. with the extra step of removing the lang attribute too since I saw that svelte-preprocess actually checks the attribute before preprocessing it. Just to make sure, is this step necessary too?

[dummdidumm]

I also did something similar for my library. with the extra step of removing the lang attribute too since I saw that svelte-preprocess actually checks the attribute before preprocessing it. Just to make sure, is this step necessary too?

That's a good catch. If the preprocessed output still contains the lang attribute it would be preprocessed again.

[firefish5000]

I also did something similar for my library. with the extra step of removing the lang attribute too since I saw that svelte-preprocess actually checks the attribute before preprocessing it. Just to make sure, is this step necessary too?

That's a good catch. If the preprocessed output still contains the lang attribute it would be preprocessed again.

Would this also happen if we enable defaults in svelte-preprocess? I prefer to specify lang rather than using defaults as well, but I am curious if the preprocessor gets triggered when a user set their defaults to pug, ts, and sass and then installs pure svelte components.

[dummdidumm]

I guess it will. Maybe the default settings hurt more overall than they help..

[firefish5000]

I guess it will. Maybe the default settings hurt more overall than they help..

I wouldn't say they are the issue. When I enable a preprocessor, I expect it to run on my code that I wrote in my current project. Touching code anywhere else is a taboo which I should need to explicitly enable. The issue is, atm, it seems that taboo is the default behavior, non-optional, and for some people it is even something that got forced down their throats because they wanted to install some component that used a preprocessor.

While this script is certainly a necessity that will finally stop adding converts to our cult and give the devs a chance their throats with mouthwash. Perhaps we still need a bit more to change with the ecosystem before it will become possible to achieve a full and proper solution to our problem. This conversation leaves me wondering

1. Is it currently possible to limit preprocessing to certain paths (not just in the cli, but in our bundler as well)?
   a. If not, is there a reason not to?
   b. Are there other ways to handle this? What do other languages do?
2. Is it likely that running preprocessors on pure svelte files will cause issues in practice (and are we already doing that)?

For the first question, I know svelte.preprocess gives at least a filename to the preprocessors. If it gave the full path, then there would be a way to resolve this delima in the individual preprocessors.
As for methods of resolving this issue... to the best of my limited knowledge, most preprocessed languages offer options akin to src/out file/dir and include/exclude globs. The only other option I know of is to use a different file extension...
And now I am thinking that was the intent of giving us a filename. pngwn may have thought this through even better than the official preprocessor, as mdsvex only processes .svx extension by default.

Now, does re-running any of the preprocessors seem likely to cause issues in practice (other than wasted cycles)? I would imagine so, though it depends on what preprocessor and what plugins are enabled by the end dev. For instance:

1. Postcss with the sugarss parser will fail miserably at parsing any valid css, as sugarss does not permit brackets nor semicolons.
2. I can't imagine pug working at all unless the preprocessor is smart enough to do nothing when pug syntax isn't used (but as a dev I would expect an error to be thrown).

Typescript might be fine, since (iirc) type checking isn't done by svelte-preprocess, and js is valid ts under normal circumstances.

Both of those cases should be possible to test now using either this script, or by enabling defaults in svelte-preprocess and installing any pure-svelte component. I have not tried either of them yet, so let me know if you do.

TL;DR
We have an unresolved issue in regards to svelte-preprocess that will likely cause issues even with this script. The issue being it can run on pure svelte files that are outside the project we are in. The solution needs to work when svelte.preprocess is called by both this script and rollup/webpack.
To fix this, we could modify svelte.preprocess to enable a include/exclude: Array<Glob> or similar config field to limit preprocessing to specific files. With an exclude field that blocks at least node_modules by default.
Or we could make svelte-preprocess only process specific extension so it doesn't run on pure svelte files. This was probably the intent of whoever wrote svelte.preprocess if the filename is the only extra context it gives you.

[dummdidumm]

cc @kaisermann who is the author of svelte-preprocess.

The fact that svelte-preprocess does run on normal svelte files is the desired behavior. It does check the lang/type attribute to know what to preprocess in which way. defaultLanguages makes this a problem because then it's always run. Making svelte-preprocess run only on non-svelte files is a no-go for me. Some kind of include/exclude seems good.

[firefish5000]

Making svelte-preprocess run only on non-svelte files is a no-go for me. Some kind of include/exclude seems good.

Sounds good to me.
New question, does the script need to standardize extensions to .svelte? Taking mdsvex's .svx back into account, this script (and mine) currently just changes the directory and keeps the filename. But I believe a standard svelte installation will only look at .svelte files. If it should mangle the extensions, easiest option I see is to tack on .svelte, and throw on conflict (like src/Component.svx and src/Component.svx.svelte both exists in src).

[kaisermann]

Those exclude/include props would only function as filters, right? The exclude option is very straight-forward to grasp: makes svelte.preprocess skip preprocessing a filename that matches one of the globs. What would include mean?

[dummdidumm]

include would mean the opposite: only preprocess files that match the glob. Starting with exclude would probably be enough.

[firefish5000]

Starting with exclude would probably be enough.

Agreed, include is just a convenient way to exclude everything else in our case. Totally unnecessary, could even be more confusing to have it since improper usage would easily conflict with svelte's extensions option.