StardewSpeak

Play Stardew Valley by voice.

This mod is primarily made for people with limited use of their hands who want to play Stardew Valley without interacting with the keyboard or mouse.

Installation

StardewSpeak can be installed just like any other SMAPI-based mod. Download the latest release and unzip its contents into your Stardew Valley Mods folder.

Test out your microphone to ensure that it is able to clearly capture your speech. The speech engine uses about 2.5 gigabytes of RAM, so be careful about running it alongside other RAM-intensive applications.

Currently, StardewSpeak is only usable on Windows 10.

Getting Started

Speech recognition begins automatically when Stardew Valley is launched. If you are playing in windowed mode, the microphone icon in the taskbar indicates that speech recognition is active. Additionally, an in-game notification will appear.

All menus can (or will) be navigated by voice. As an example, to load a saved game from the title menu, say load to enter the saved game menu, then game followed by the number of the game you want to play, e.g. game three to load the third game. If you have more than four saved games, scroll down and scroll up will click the up and down arrows. See the menus file for more information and a list of available menu-specific commands.

Once in game, try saying go to farm to begin walking from your farmhouse interior to your farm. Saying stop will stop whatever your farmer is currently doing.

Development

Documentation for developers can be found here.

Commands

Commands wrapped in brackets are optional, meaning that hello [world] will match either hello or hello world. Commands wrapped in brackets or parentheses with | describes alternatives. Commands wrapped in <> refer to a particular set of alternatives: <direction> refers to movement directions, <n>, <x>, and <y> refer to numbers, <location> refers to game locations, and <item> refers to game items.

General

Command	Description	Example(s)
<direction>	Begin moving in a specific direction. Options are north (up), east (right), south (down), west (left), main (up and right), floor (down and right), air (down and left), and wash (up and left). Using a mnemonic with USA states: Maine in the northeast, Florida in the southeast, Arizona in the southwest and Washington in the northwest.	"north"
<direction> <n>	Move n tiles in a direction and stop. Will pathfind around obstacles as long as the target tile is clear.	"one two west" - move left 12 tiles
navigate <direction>	Combines the two above commands. Begin moving in a specific direction while pathfinding around terrain.	"navigate east"
face <direction>	Face direction.	"face east"
clear (debris | stones | rocks | twigs | wood | weeds)	Begin clearing weeds, stone, or wood. Saying clear debris will clear all types.	"clear debris" "clear wood"
(clear | mine) ore	Begin mining ore and gem nodes.	"clear ore"
clear grass	Begin clearing grass.	"clear grass"
chop trees	Begin chopping down nearby trees.	"chop trees"
go to <location>	Walk towards a game location.	"go to mines"
(go to mail box | (check | read) mail)	If on the farm, go the the mailbox and press the action button.	"go to mail box" "check mail"
go to shipping bin	Go to the shipping bin and press action button.	"go to shipping bin"
go to billboard	If in town, go to the billboard and press action button.	"go to billboard"
go to calendar	If in town, go to the calendar and press action button.	"go to calendar"
(dig | hoe) <x> by <y>	Use hoe to dig an x by y grid based on the last two directions faced.	"dig three by four"
start planting	Start planting equipped seeds or fertilizer on available hoe dirt.	"start planting"
water crops	Start watering nearby crops.	"water crops"
harvest crops	Start harvesting fully grown crops.	"start harvesting"
pet animals	Attempt to pet all animals in the current location. Will sometimes fail if the animals are clumped together or are in tight areas that make pathfinding difficult.	"pet animals"
milk animals	Attempt to milk all cows and goats in the current location. Will sometimes fail if the animals are clumped together or are in tight areas that make pathfinding difficult.	"milk animals"
start fishing	Cast fishing rod at maximum distance. If the cast is successful, wait for a nibble and begin reeling.	"start fishing"
catch fish	Automatically complete fish catching minigame. Will also catch any treasure chests that appear.	"catch fish"
talk to <npc>	Move to an NPC and press action button. If the player is holding a giftable item this will gift that item to the NPC. Will fail if the NPC is not in the current location.	"talk to Leah"
start shopping	If in a store location (Pierre's General Store, Marnie's house, etc.), move to shopkeeper and press action button.	"start shopping"
[open | read] (quests | journal | quest log)	Open journal.	"read journal"
go inside	Go inside the nearest building, including farm buildings.	"go inside"
nearest <item> [<n>]	Move to nearest item by name in current location. If n is specified, go to the nth closest item.	"nearest chest" "nearest bee house three"
(action | check)	Press action button (default x)	"action"
swing	Use tool (default c)	"swing"
start swinging [tool]	Hold down use tool button (default c)	"start swinging"
stop	Stop current actions.	"stop"
item <n>	Equip the nth item in the toolbar.	"item seven"
equip <item>	Equip item if in inventory.	"equip pickaxe" "equip cauliflower seeds"
equip [melee] weapon	Equip melee weapon if in inventory.	"equip weapon"
go to ladder	Move to the ladder up in the mines and press the action key.	"go to ladder"
go to elevator	Move to the elevator in the mines and press the action key.	"go to elevator"
ladder down	Move to the nearest ladder down in the mines or skull cavern if at least one has been revealed and press the action key.	"ladder down"
attack	Move to the nearest monster in the current location and begin swinging melee weapon. Continue until there are no more monsters in the current location.	"attack"
defend	Wait until a monster comes into melee weapon range and swing at it. Continue until there are no more monsters in the current location.	"defend"
(next | cycle) toolbar	Cycle the toolbar.	"next toolbar"
skip [cutscene | event]	Skip the current cutscene.	"skip cutscene"
[left] click [<n>]	Left click n times (default one).	"click"
right click [<n>]	Right click n times (default one).	"right click"
mouse <direction> [<n>]	Move the mouse n tiles (64 pixels). Ideally, this command and the one below will only be used sparingly when more specific commands are unavailable.	"mouse down"
small mouse <direction> [<n>]	Move the mouse n pixels.	"small mouse down seven"
hold mouse	Hold down the left mouse button.	"hold mouse"
release mouse	Release the left mouse button if it is being held.	"release mouse"