Bash is great for automating little tasks, but sometimes a little script you think will take a minute to write turns into a half hour or more. This is the story of one of those half-hour scripts.
I have too many 3D printers. In particular, I have three that are almost — but not exactly — the same, so each one has a slightly different build process when I want to update their firmware. In all fairness, one of those printers is heading out the door soon, but I’ll probably still wind up building firmware images for it.
My initial process was painful. I have a special directory with the four files needed to configure Marlin for each machine. I copy all four files and ask PlatformIO to perform the build. Usually, it succeeds and gives me a file that looks like firmware-yyyyddmmhhmm.bin or something like that.
The problem is that the build process doesn’t know which of the three machines is the target: Sulu, Checkov, or Fraiser. (Long story.) So, I manually look at the file name, copy it, and rename it. Of course, this is an error-prone process, and I’m basically lazy, so I decided to write a script to do it. I figured it would take just a minute to bundle up all the steps. I was wrong.
Copying the files to the right place was a piece of cake. I did check to make sure they existed. The problem came from launching PlatformIO, seeing the result on the screen, and being able to parse the filename out of the stream.
I thought it would be easy:
That should do the build and leave $FN with the name of the file I need to rename and process. It does, but there are two problems. You can’t see what’s happening, and you can’t tell when the build fails.
The pipeline consumes the build’s output. Of course, a tee command can manage that, right? Well, sort of. The problem is that the tee command sends things to a file and standard out, but the standard out, in this case, is the pipe. Sure, I could tee the output to a temporary file and then process that file later, but that’s messy.
So, I resorted to a Bash-specific feature:
This puts the output on my screen but still sends it down the pipe, too. Sure, there are cases when this isn’t a good idea, and it isn’t very portable, but for my own use, it works just fine, and I’m OK with that. There are other ways to do this, like using /dev/tty if you know you are only using the script from a terminal.
The bigger problem is that if the build fails — and it might — there isn’t a good way to fail the whole pipeline. By default, the pipe’s return value is the last return value, and cut is happy to report success as long as it runs.
There are a number of possible answers. Again, I could have resorted to a temporary file. However, I decided to set a bash option to cause any failing item in a pipe to fail the whole pipe immediately:
So now, in part, my script looks like this:
Is it brain surgery? Nope. But it is one of those bumps in the road in what should have been a five-minute exercise. Maybe next time you run into it, you’ll save yourself at least 25 minutes. This gets the job done, but it isn’t a stellar example of bash programming. I would hate to run it through a lint-like checker.
So, stop using Marlin?
Maybe, but the real idea is how to manage something failing in the middle of a pipeline which comes up all the time. Not really a Marlin/not Marlin issue.
Hey, my name’s Dave and I run a channel online where I go over bash tips and tricks. I see a lot of ways this code can be cleaned up.
For starters, you can get the exit code of every command in a pipeline with the builtin array variable in bash PIPESTATUS. For example:
$ echo hi | grep hi | grep bye | tail
$ echo "${PIPESTATUS[*]}"
0 0 1 0
We can quickly identify that the 3rd command in this pipeline failed.
But beyond that, you don’t actually need a complicated pipeline and can break it up into smaller chunks of work. You can rework your whole script:
#!/usr/bin/env bash
# run command, save stdout and exit code
output=$(pio run)
code=$?
# print all command output
echo "$output"
# stop here if the command failed
if ((code != 0)); then
echo ‘Build failed’
exit 3
fi
# command was a success, extract filename and copy it
filename=$(awk ‘/^Renamed to/ { print $3 }’ <<< “$output”)
# stop here if awk failed or the filename variable is empty
if (($? != 0)) || [[ -z $filename ]]; then
echo ‘Failed to extract build filename’
exit 3
fi
# we are good to go
echo ‘Success…’
cp “.pio/build/STM32F103RC_creality/$filename” “configurations/$1” || exit 3
echo “Result: configurations/$1/$filename”
“`
This will:
1. add more error checking
2. print the `pio run` output to the screen and retains its exit code
3. parse the filename with a single invocation of `awk` (and error check it)
Bash can be super fun and I love seeing posts about it on this site!
dave
No doubt there are many ways to do this. While $PIPESTATUS clearly would work, I didn’t like the idea of using output=$(…) for the same reason I didn’t want to use temporary files. There is a ton of output from the build. Granted, this is a quick one-off so it didn’t really matter but that’s kind of the point, too, is it is a one-off. I would have been more fastidious with errors had I wanted to do this in a real environment. As it was, it was just a quick-and-dirty way to be lazy 😉 If you look back in the Linux Fu archives, there are a lot of bash tricks ranging from the silly to the sublime 😉
If you put your bash code into a file it is never a one-off but a solution for eternity. =)
So a little effort to make it serviceable is time well spent. I have such simple helpers that are more than a decade in use.
And I’m with Dave on this one: catching intermediate output and breaking up a pipe is often the right thing to do. Especially if one needs to react to complex situations.
I agree. He was trying to put way too much code in one line and then complaining about the obvious consequences of doing that.
“I have too many 3D printers.” Said no-one, ever.
It does make me wonder how many Al’s running, hey. Is it really only 3? Or is it only 3 that are largely identical, with additional non-identical machines rounding out the total?
3 workhorse FDMs. Two other FDMs. Two Resin. 1 laser cutter. 1 light duty CNC….
So, I’m guessing that Fraiser will be the one getting the walking papers.
(Because its name is not associated with the other two.)
Any particular reason you don’t just use PlatformIO’s pre- and post-build actions to do this? Presumably, the pre action could handle which machine you are doing the build for…
Documentation on PlatformIO’s scripting actions
Or, don’t care that it failed? Either you found a renamed thingiemabob that is valid, build passed, or it’s empty, build failed.
Alternatively, spitballing on my phone, something like
`FN=”$(pio run || exit_fun 1 | grep | cut)”`
might work? No clue, never tried it; but bshellcheck seems to be ok with it, though it just might not do what you expect too :p
This should call your exit function (or just `exit 1` if you don’t need the result)
I have a rule to only write POSIX-compliant scripts and don’t use bashisms, so maybe there’s a more “bashy” way to do this better.
There is nothing bash specific about /dev/fd/2.
That’s provided by the OS.
Keptin! I don’t trust Fraiser!
… Steady as she goes, Mr. Chekov.
[Fraiser adjusts his red shirt awkwardly] I have a bad feeling about this.
… Don’t mix universes, Mr. Fraiser. Never quote me the odds. You’ll be fine.
[10 minutes later]… He’s dead, Jim.
Thank you all! I love seeing the alternative ways of doing things (and the nerd jokes are sublime)!
ChicagoVPS is your gateway to unparalleled hosting solutions. Our state-of-the-art datacenters and powerful network ensures lightning-fast speeds and uninterrupted connectivity for your websites and applications. Whether you’re a startup looking for scalable resources or an enterprise in need of enterprise-grade hosting, our range of plans and customizable solutions guarantee a perfect fit. Trust in ChicagoVPS to deliver excellence, combining unmatched reliability and top-tier support.
For Inquiries or to receive a personalized quote, please reach out to us through our contact form here or email us at sales@chicagovps.net.
