My First Open Source Tool
“Often, because of necessities given by my work, I happen to write Bash scripts. They’re quite simple, I’m no expert, and I tend to automate simple tasks. I work in backup and data protection fields, so I often copy files, check integrity, calculate times, etc. But in the last weeks, for the first time, I wrote a script that I wanted to share with the world: not because it’s super cool –at all–, but because nobody else provided a tool for this purpose.
⚠️ This will be a quite technical post, you may get bored. ⚠️
Nobody Else Will Be There 🎵
A brief introduction to better explain the scenario.
As some of you may know, many companies in several areas rely on magnetic tapes to store their backup data. As audio cassettes and VHS were read by specific recorders, so are enterprise storage tapes. The tapes are often organized in large libraries (called no surprisingly Tape Libraries) that can contain more than 2,000 elements and dozens of Tape Drives, the ones that read and write data when cartridges are mounted. The most widely-used standard is called Linear Tape-Open, or LTO. Every three or four years, a new generation of LTO tapes and drives is published: using the same format, the capacity of a single tape is doubled, if not more. For example, LTO-1 could store 200 GB, and LTO-9 can hold 45 TB! 🤯
For years, LTO tapes have been amazing: quite affordable and reliable, they’ve been a more than valid solution for long-term backups. LTO-9 technology should have been launched in 2020, right when the pandemic hit. There had been delays, but eventually it arrived in late 2021 — right when the semiconductor shortage hit. But. There was a marginal note, when the first libraries were delivered to the customers: each LTO-9 tape has to be optimized1 the first time it is mounted in any tape drive.
“Ok, so what?” one could ask.
When reading one of the several documents explaining the process, it becomes clear that the optimization of a tape requires from twenty minutes up to two hours for each single tape. So, when a “virgin” LTO-9 tape is loaded in a tape for the first time, the optimization process starts automatically. The tape drive becomes unresponsive, and the only option is waiting.
“Ok, but what’s the problem?” is the next question.
The problem is that some customers buy very large stocks of tapes, even more than 1,000. So, to prepare all those tapes, in the worst case, it will take 1,000 * 2 hours = 2,000 hours.2 Whaaaat?!
“Ok, but… Don’t the Tape Library vendors provide an automated tool to perform the optimization in the background?”
That’s the point: no, they don’t.3
Angry Chair 🎵
Personally, I think that it is not fair and modern Tape Libraries should have a feature like this, especially two years after the technology came out. So, because of my job, I was obliged to write a script to avoid to manually move all the cartridges in the tape drives, one by one. By chance, I had the opportunity to work on two different enterprise Tape Libraries, both from Quantum and IBM, and both of them had a gazillion tape drives. Then I thought: it would be nice to parallelize and automate the load/unload of the several drives.
And then, after a couple of tentatives, I was able to find a quite elegant(?) solution with just Linux tools (mtx).
I tested it on both libraries and… it worked better than expected!
Occasionally, it still “loses control” of a drive, but it’s not a big issue: it just needs to be relaunched.
Then, I decided to publish it on a public repository, so any sysadmin with this particular need can use it. I know it is for a very specific problem, but it also avoids a big pain… but I still can’t realize that I had to write it: as I already said, it should have been built-in in the expensive Tape Libraries.
Maybe I took this matter too much to heart, but I hope that this simple script can be useful to even just one person. As the meme says, it ain’t much, but it’s honest work.
🎮 Alan Wake Remastered (almost finished!)
🎧 Red Eyes Unravelling by Flies Are Spies From Hell
📖 The October Country by Ray Bradbury
-
Each vendor uses a different term: calibration, optimization, … but they refer to the same process. ↩︎
-
From my experience, the average requested time is 30–40 minutes. ↩︎
-
As far as I know, in July 2023, only one of the main vendors has a built-in function to provide this procedure, but I may be wrong.” ↩︎
“Often, because of necessities given by my work, I happen to write Bash scripts. They’re quite simple, I’m no expert, and I tend to automate simple tasks. I work in backup and data protection fields, so I often copy files, check integrity, calculate times, etc. But in the last weeks, for the first time, I wrote a script that I wanted to share with the world: not because it’s super cool –at all–, but because nobody else provided a tool for this purpose.