Our community never fails to impress with clever, creative, and downright fun buildsâand this one might just take the cake (block?). Recently, weâve been talking about how Stream Deck is showing up in more places than ever: with Stream Deck Modules and Virtual Stream Deck on your computer. But now? Itâs in Minecraft.
Yep, thatâs right. A fully functional Stream Deckâinside Minecraft.
This project was created by community member TheAppleFreak, who turned a clever concept into an actual working setup. Hereâs how they made it happen, in their own words:
Inspiration
About a month ago, an Elgato community specialist posted a Minecraft build of a 15-key Stream Deck to the Elgato Twitter. Possessed by the spirit of cordial one-upsmanship, I began trying to figure out how to turn this concept into an actual functional reality. While what I made isn't perfect by any stretch of the imagination, and does have a tendency to break if you stare at it the wrong way, I'm more than pleased with the results!
This build is powered by a six key Virtual Stream Deck hidden surreptitiously off on my other monitor, with a very hacky Node.js script handling the hard parts of pressing SD buttons based on what's activated in game, as well as converting the VSD display into something that can be represented in Minecraft.
Whatâs happening in Minecraft
On the Minecraft side, the redstone involved is extraordinarily simple. Behind the screen, every button uses a pair of calibrated sculk sensors (in-game sound sensors) to monitor for input events, which in this case are either snowballs landing on the button (signal strength 2) or the sound of a wind charge exploding (signal strength 15). Each pair leads into a single command block that, when triggered, puts between 1 to 8 spaces in the in-game chat. The page back/forward buttons off to the sides also have a tiny bit of extra circuitry that controls the indicator lights, but otherwise function identically.
Whatâs happening in my custom script
The bulk of the magic happens in the custom Node.js script I wrote for this project. In the initial program startup, I get some special information that lets me select either a Virtual Stream Deck (VSD) window or the Minecraft game window, then also get the coordinates of the VSD window. Once all the setup is done, thereâs two loops that run in tandem with each other: input and display.
The input half of the program is the simpler of the two to reason about. Whenever Minecraft runs, it keeps a log file of most important things that happen in a text file aptly named latest.log. Importantly, this log file also includes any messages received in chat, which means with a little bit of clever code we can look at only the log entries that contain our spaces-only messages. When such a message arrives, we count the number of spaces and use that to determine which button to press. I ordered the buttons top-to-bottom, left-to-right (thereâs also two additional buttons for page back/forward). Once we get the button selected, I use a specialized desktop automation library to very quickly move the mouse out of the Minecraft window, click the corresponding button on the Virtual Stream Deck powering everything, and return it back to Minecraftâs control (you have to press F3+P to make this work; otherwise, itâll pause when the game loses focus). If youâre not moving the mouse when these moves happen, itâs pretty seamless! if you are, it tends to break horribly lol
That handles pressing buttons fine, but the real magic of the Stream Deck is that you have the programmable display under each key. How do you show that within Minecraft? The solution that I chose is perhaps not the most elegant possible, but it works, which is what I care about! Four times a second, the script takes a screenshot of the VSD window. Assuming the buttons are at a specific size (the default size of 52x52 px works quite well), it crops the screenshot to only the size of a button, then resizes it to 8x8 px large. Once thatâs done, it goes over each pixel in the cropped image and runs them through some clever code to figure out what block matches its color the closest. (Many thanks to kult0922 on GitHub, who made a website that converts images to Minecraft pixel art and published the source code for it! Without that code, this project likely would have taken a good bit longer to figure out). Just make sure to rinse and repeat for the other five buttons
Getting the list of blocks youâd need is a good step, but getting them into the game is still a problem. If you have a way to run chat commands, you could use the /setblock command a bunch to update each pixel one at a time, but thatâs very slow and very inefficient. A better tool to use would be the Structure Block, a tool in the vanilla game that among other things lets you place premade âschematicsâ of structures into the game world. I figured out how to have my script create schematics in the way that Minecraft was expecting them, and once they get saved in the schematics folder I can issue a chat command that tells the Structure Block in game to look at the correct schematic for the current state of the screen (each individual frame needs to have a unique filename, which complicates things a little bit). Once thatâs done, you just have a tiny redstone circuit repeatedly paste the active schematic into the world, and youâve got a working screen!
Conclusion
At the end of the day, youâve got a mostly usable Stream Deck, but in Minecraft! While my script is designed primarily to run on my machine only, I donât see any reason why it canât be improved to maybe run on multiplayer servers! The original prototype of this project actually ran on an instance of Stream Deck Mobile running in Windows Subsystem for Android, which technically means you could run everything on a completely separate device or in a virtual machine. Whether thatâs a good idea is an entirely different question, of course⊠but donât let concerns like âpracticalityâ stop you from playing around :)
Fun fact: the Stream Deck logo at the top was made using 24 maps. The logoâs structure is located in the End void, but itâs so big that it stretches nearly from one end of the void to the other.
Check out more of TheAppleFreakâs work:
A CIKKBEN SZEREPLĆ TERMĂKEK