Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

PoseNet Bug using custom trained model in a reactjs environment #1468

Open
yehonatanyosefi opened this issue Jun 19, 2023 · 6 comments
Open

PoseNet Bug using custom trained model in a reactjs environment #1468

yehonatanyosefi opened this issue Jun 19, 2023 · 6 comments

Comments

@yehonatanyosefi
Copy link

Hey there!
I'm trying to use ml5js in a react poseNet project i'm doing and even though i use the same inputs as in a similar yoga project i get this error when doing:
let inputs = []
for (let i = 0; i < pose.keypoints.length; i++) {
let x = pose.keypoints[i].position.x
let y = pose.keypoints[i].position.y
inputs.push(x)
inputs.push(y)
}
brainRef.current.classify(inputs, gotResults)
it somehow doesn't recognize the right inputs in the latest ml5js library and the results on gotResults are undefined.

error:
TypeError: Cannot convert undefined or null to object
    at Function.keys (<anonymous>)
    at t.<anonymous> (index.js:1044:1)
    at l (runtime.js:63:1)
    at Generator._invoke (runtime.js:294:1)
    at http://Generator.next (runtime.js:119:1)
    at n (asyncToGenerator.js:3:1)
    at s (asyncToGenerator.js:25:1)
    at asyncToGenerator.js:32:1
    at new Promise (<anonymous>)
    at t.<anonymous> (asyncToGenerator.js:21:1)

it says the error is in classifyInternal on the npm package (i'm using latest one).
would you please be able to help?
@yehonatanyosefi
Copy link
Author

yehonatanyosefi commented Jun 27, 2023 via email

@lindapaiste
Copy link
Contributor

React stack traces are the worst. All I can tell is that somewhere inside classifyInternal (or any of the functions that it calls) there is an Object.keys(something) on a something which is not an object.

The only top-level Object.keys are on meta.inputs and meta.outputs. I don't know how those could be undefined but humor me and add some extra logging. The line we're looking for might be something else really deep in the tree and hard to find.

const classifyPose = async () => {
	try {
		if (pose && skeleton.length) {
			let inputs = []
			for (let i = 0; i < pose.keypoints.length; i++) {
				let x = pose.keypoints[i].position.x
				let y = pose.keypoints[i].position.y
				inputs.push(x)
				inputs.push(y)
			}
			console.log('Inputs', inputs);
			const meta = brainRef.current.neuralNetworkData.meta;
			console.log('Meta', meta);
			console.log('meta.inputs', meta.inputs);
			console.log('meta.outputs', meta.outputs);
			const results = await brainRef.current.classify(inputs);
			gotResults(undefined, results);
		} else {
			// console.log('Pose not found')
			setCurrPose('Not found')
			posesArray.current = [...posesArray.current, null]
		}
		handleGameTik()
	} catch(e) {
		console.log('Caught error', e);
		console.trace();
	}
}

@yehonatanyosefi
Copy link
Author

yehonatanyosefi commented Jul 2, 2023 via email

@lindapaiste
Copy link
Contributor

Okay we are getting somewhere! You’ve confirmed that meta.inputs and meta.outputs are both undefined and that’s what triggers the TypeError.

In looking at your meta object I can see that the info which we need is there but it’s not in the right place. It looks like it’s meta.meta.inputs instead of meta.inputs.

I haven’t got to the root problem yet. That is, why the meta is structured incorrectly. I’ll need to play around more and run your code. It’s either a problem with reading your metadata.json file or it’s a problem with exporting the model which led to an incorrect metadata.json. There probably exists a quick fix where I tell you what to change in the metadata.json file to make it work but that’s a poor solution that doesn’t address why it’s wrong. Did you use ml5 to generate the saved model that you are loading?

@yehonatanyosefi
Copy link
Author

yehonatanyosefi commented Jul 2, 2023 via email

@yehonatanyosefi
Copy link
Author

yehonatanyosefi commented Jul 3, 2023 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lindapaiste @yehonatanyosefi