Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly hash images based on contents and allow skipping generation of images #826

Open
wants to merge 17 commits into
base: v0
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
215 changes: 215 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

29 changes: 25 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,27 @@
# HEY

This is a fork, to fix the fingerprinting of static files. This package should in fact use the contents of the file to create the `[hash]` part of static filename.

I couldn't work out the best way to report back this change to the main module as they are preparing for a new release, so it exists here, for you to:

## Install

```
yarn add jthawme-nuxt-image // or npm install jthawme-nuxt-image
```

With nuxt 2 you also have to add this package to the `transpile` property in `nuxt.config.js`

```
...
build: {
// other config etc.
transpile: ["jthawme-nuxt-image"]
},
```

---

[![@nuxt/image](./docs/public/cover.jpg "Nuxt Image")](https://image.nuxtjs.org)

[![npm version][npm-version-src]][npm-version-href]
Expand All @@ -8,21 +32,18 @@
- [📖  Read Documentation](https://image.nuxtjs.org)
- [▶️  Play online](https://githubbox.com/nuxt/image/tree/main/example)


### Contributing

1. Clone this repository
2. Install dependencies using `yarn install`
3. Start development server using `yarn dev`


## 📑 License

Copyright (c) Nuxt Team



<!-- Badges -->

[npm-version-src]: https://flat.badgen.net/npm/v/@nuxt/image
[npm-version-href]: https://npmjs.com/package/@nuxt/image
[npm-downloads-src]: https://flat.badgen.net/npm/dm/@nuxt/image
Expand Down
7 changes: 4 additions & 3 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
{
"name": "@nuxt/image",
"version": "0.7.0",
"name": "jthawme-nuxt-image",
"version": "0.8.7",
"description": "Nuxt Image Module",
"repository": "nuxt/image",
"repository": "https://github.com/jthawme/nuxt-image-fork",
"license": "MIT",
"sideEffects": false,
"main": "./dist/module.cjs",
Expand All @@ -19,6 +19,7 @@
"lint": "eslint --ext .ts --ext .vue .",
"prepack": "yarn build",
"release": "yarn test && standard-version && git push --follow-tags && npm publish",
"release:no-test": "standard-version && git push --follow-tags && npm publish",
"test": "yarn lint && yarn jest --forceExit",
"test:e2e": "jest test/e2e --forceExit",
"test:unit": "jest test/unit --forceExit"
Expand Down
36 changes: 31 additions & 5 deletions src/generate.ts
Original file line number Diff line number Diff line change
@@ -1,15 +1,27 @@

import { createWriteStream } from 'fs'
import { createWriteStream, existsSync } from 'fs'
import { promisify } from 'util'
import stream from 'stream'
import path from 'path'
import { mkdirp } from 'fs-extra'
import { dirname, join, relative, basename, trimExt } from 'upath'
import { fetch } from 'node-fetch-native'
import { joinURL, hasProtocol, parseURL, withoutTrailingSlash } from 'ufo'
import hasha from 'hasha'
import pLimit from 'p-limit'
import { ModuleOptions, MapToStatic, ResolvedImage } from './types'
import { hash, logger, guessExt } from './utils'

function getHash (input: string, url: string) {
const staticFile = path.join(process.cwd(), 'static', input)

if (existsSync(staticFile)) {
return hash(`${hasha.fromFileSync(staticFile)}${url}`)
}

return hash(url)
}

export function setupStaticGeneration (nuxt: any, options: ModuleOptions) {
const staticImages: Record<string, string> = {} // url ~> hashed file name

Expand All @@ -21,7 +33,7 @@ export function setupStaticGeneration (nuxt: any, options: ModuleOptions) {
const params: any = {
name: trimExt(basename(pathname)),
ext: (format && `.${format}`) || guessExt(input),
hash: hash(url),
hash: getHash(input, url),
// TODO: pass from runtimeConfig to mapStatic as param
publicPath: nuxt.options.app.cdnURL ? '/' : withoutTrailingSlash(nuxt.options.build.publicPath)
}
Expand All @@ -33,6 +45,8 @@ export function setupStaticGeneration (nuxt: any, options: ModuleOptions) {
})

nuxt.hook('generate:done', async () => {
const images = options.fetchImageDictionary ? await options.fetchImageDictionary(staticImages) : []

const limit = pLimit(8)
const downloads = Object.entries(staticImages).map(([url, name]) => {
if (!hasProtocol(url)) {
Expand All @@ -41,19 +55,31 @@ export function setupStaticGeneration (nuxt: any, options: ModuleOptions) {
return limit(() => downloadImage({
url,
name,
outDir: nuxt.options.generate.dir
outDir: nuxt.options.generate.dir,
skip: options.skipGeneration,
imageDictionary: images
}))
})
await Promise.all(downloads)
})
}

const pipeline = promisify(stream.pipeline)
async function downloadImage ({ url, name, outDir }: { url: string, name: string, outDir: string }) {
async function downloadImage ({ url, name, outDir, skip, imageDictionary }: { url: string, name: string, outDir: string, imageDictionary: string[], skip?: ModuleOptions['skipGeneration'] }) {
try {
const dstFile = join(outDir, name)

if (skip) {
const shouldSkip = await skip(name, fetch, imageDictionary).then(shouldSkip => shouldSkip)

if (shouldSkip) {
logger.success('Skipping generation for ' + relative(process.cwd(), dstFile))
return
}
}

const response = await fetch(url)
if (!response.ok) { throw new Error(`Unexpected response ${response.statusText}`) }
const dstFile = join(outDir, name)
await mkdirp(dirname(dstFile))
await pipeline(response.body as any, createWriteStream(dstFile))
logger.success('Generated static image ' + relative(process.cwd(), dstFile))
Expand Down
2 changes: 2 additions & 0 deletions src/types/module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ export interface ModuleOptions extends ImageProviders {
screens: CreateImageOptions['screens'],
internalUrl: string
providers: { [name: string]: InputProvider | any } & ImageProviders
skipGeneration?: (destFile: string, fetch: any, imageDictionary: string[]) => Promise<boolean>,
fetchImageDictionary?: (staticImageDictionary: Record<string, string>) => Promise<string[]>,
[key: string]: any
}

Expand Down
2 changes: 1 addition & 1 deletion src/utils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ export const logger = consola.withScope('@nuxt/image')

export const pkg = { name, version }

export function hash (value: string, length = 6) {
export function hash (value: string, length = 32) {
return hasha(value).substr(0, length)
}

Expand Down