Servo based systems are both incredibly power and incredibly fast. So fast that if you've never seen a servo based arm move at top speed and come to that top speed at top acceleration you'd be forgiven to think that I'm exaggerating.
Depending on the application there are different layers of safety surrounding these systems, including perimeter guards, optical barriers, limit switches, resistance based detection and so on. But when a system is broken someone has to go in and fix it, and you tend to do that with the robot powered up, some of the safety systems disabled so you can actually work on it and if you're really unlucky a motor will end up shorted against a + or - rail while you're within reach. This is obviously dangerous, and it is more dangerous because broken equipment can't be trusted to behave in a predictable way.
They won't stop. Not until whatever is obstructing has moved or the motor has burned out (or someone has the presence of mind to hit the e-stop). I've seen a 3" thick mount that must have weighed well over a ton sheared clear of its bolts (which themselves were an inch thick) by a malfunctioning servo on a very large lathe under construction (think 8 foot chuck for crane cable idler wheels). Do not fuck with servo systems unless you are 100% sure they are safe to approach or you may well end up dead or gravely injured.
I just want to echo the 'so fast you'd be forgiven to think that I'm exaggerating'. I work with industrial arms every day, but pretty much never with them running at full acceleration up to the speed limit between the extents of reach. Even when they bump you at a speed that would be like a mild shove from a person, the unyielding force is remarkable, and if you're ever near one and surprised by a sudden 'full send' motion, it's terrifying.
Terrifying is the proper term. The only time I've ever been really mad with a co-worker is when they decided to do a demo to potential customers with the newest (pre-release) firmware of a massive lathe without my very explicit permission. That could have well ended up with people seriously injured, as it was - of course - it happened to have a pretty bad bug in it but besides people getting scared it didn't cause any damage or injury.
My own process to ensure things were safe was to first run it in air without the spindle powered up, then on foam, chalk, and finally on aluminum in that order before progressing to steel.
I had the pleasure of touring BMW’s Welt manufacturing plant while I was there and it was the first time I’d seen manufacturing robots in action. Even standing well outside of their reach zones and behind at least two layers of fencing, their movement was terrifying. Precise, powerful and completely without any awareness of their surroundings. They remind me of horses to be honest.
All of that is to say that it seems like working on a manufacturing floor with robots has the potential to be so so dangerous. Getting into a rhythm and doing repetitive work all day, it’s easy to imagine slipping and ending up as human soup.
There's a catch 22 when repairing machinery, you have to see it in action to determine what is going wrong, and sometimes this means getting up close with it during operation. I've seen many techs/mechanics bypass guards and not use lockouts when diagnosing machinery problems. And of course everyone is "ok" with it because they are supposed to be experts and know what they are doing.
> I thought OSHA and friends didn't allow this. Lockout/tagout is standard.
Yes it is, for plant workers it is a firing offense in many places.
But your typical maintenance tech that is supposed to work on the machinery itself rather than just to be using it may well use lockout/tagout to ensure that the machine is powered off when they are working on it and don't need to be able to do any kind of diagnostics. But short of a complete disassembly and testing each component in isolation - for which there often isn't really time and which given the pressure on maintenance technicians to get a piece of gear working again - people tend to take shortcuts such as to hook up analytics gear to a machine that is live. Stupid? Yes, absolutely. But this kind of stupidity is the result of usually many years of things working just fine and bad practices creep in. The guy that manages to get stuff done rapidly is the one that gets called out. And eventually an accident will happen. Weirdly enough even near accidents tend to reinforce the belief that it worked. When actually the lesson should be that it didn't but the person just got lucky.
One very memorable occasion at an industrial plant I visited is that someone got beaten up for removing a lock and I felt absolutely no sympathy for that guy. He could have well gotten someone killed (large 5 axis mill).
Similar humbling experience for me a couple of decades ago. I made a mistake when programming the toolpath to cut a large aluminum part on our Haas VF-3SS machine. I didn't know about a bug in the CAM software and trusted the simulator.
The machine plunged into a large block of aluminum at full rapid speed with a 1 inch rougher end-mill. It then proceeded to run the cutting program, again, at full rapid rate. The sound it made is something that I will never forget. This thing was churning aluminum like it was butter. It was literally turning it into goo, not cutting it at all. It did not care one bit. The cutter was large enough and stubby enough that it did not break.
These things are fast enough that, by the time your shocked brain is able to respond it could have killed and dismembered you five times before you can even think of slamming the e-stop button.
The end-mill came out with a ball of aluminum, 2 inches in diameter, permanently welded to it. It was one of the scariest industrial accidents I've been around. Thankfully the enclosure and rigidity of the machine made it so that the only consequence was aluminum that looked like churned butter.
Years later I worked with an aerospace client who uses stir welding to assemble rockets. Same process, except you are doing it on purpose and with controls. The idea is that stir welding preserves the metallurgical properties of the material, whereas TIG/MIG welding might not preserve properties important to the design.
To echo your other comment, since that happened I learned to never trust CAM software or simulations, always run it on air first and, if necessary, machinable wax blocks. Decades later, I still don't consider myself a machinist at all. I am pretty decent at it. It's a complex art that requires time and dedication. I do too many other things to be that guy. I do have a VF-2 and a Bridgeport knee mill (full manual still) in my garage though...
It's close calls like that that are the best vaccine against getting over-confident.
I have a 30 Watt diode laser that came without any safety measures at all. It kind of surprised me that you could just fire it with a G-code and absolutely nothing stood between you and the beam. It cuts wood like butter though, up to 20 mm in a single pass (solid wood, not birch ply) and I'm super happy with it but that tool is going to go through some serious modifications to make it safe. It can set just about anything on fire fair distance away and will fry your eyeballs from half a kilometer if not more. Any idiot with a few bucks to spare can go and order one and they have absolutely no clue about the kind of damage a machine like that can do. It looks so innocent, it's completely deceptive.
>> by a malfunctioning servo on a very large lathe under construction
I got hit right in the dick by a milling machine axis lever spinning up full throttle.
Only once, never again. Still surprised something terrible didn’t happen - hurt like a bitch. Shop dude says it happens pretty often though because people don’t look down at the caution tape telling you not to stand there.
People have waaaay too much faith that motorized devices won't injure or kill them. Even motorized lie flat seats in an aircraft can probably break something. Many of these devices rely on simple current limiting. While this can help, its definately not a guarantee that it won't kill you.
Reminds me of the car salesman who put his head in an autoclosing minivan door to show off the safety sensors. Sure, it didn't kill him. But having your head pressed between two thin metal plates until current limiting detects the skull in the head is gonna suuuuck.
>One day in 1979, a robot at a Ford Motor Company casting plant malfunctioned—human workers determined that it was not going fast enough. And so 25-year-old Robert Williams was asked to climb into a storage rack to help move things along. The one-ton robot continued to work silently, smashing into Williams’s head and instantly killing him. This was reportedly the first incident in which a robot killed a human; many more would follow.
it's not even the first this year. But yeah, from https://www.bbc.com/news/world-asia-67354709 (non-paywalled coverage of the same story) it sounds very much like not doing lockout/tagout or at least hitting the e-Stop before inspecting something.
> it sounds very much like not doing lockout/tagout or at least hitting the e-Stop before inspecting something
That was exactly my thought when reading it. If the e-stop had been depressed this likely would have never happened. But with the system live all bets are off, especially if it is already malfunctioning. You're essentially one queued G-code away from getting mauled.
Article is missing some key details. Every industrial robot that I've seen has at least a line painted in the floor, or more often a physical barrier, and workers are required to stay behind it when the robot is powered on. Did this factory not have such basic precautions, or did the dead worker violate protocol, or did another worker turn on the robot at the wrong time?
Korean versions of the article have more details. He was from a robot maintenance company visiting the factory to perform a software update. He was working on the robot alone and checking the sensors when the incident happened.
Hmm, that would suggest that he both had better training about the robots (compared to a random food-packing-plant worker) but was also more comfortable around them (which is a huge risk with any powerful equipment, including automobiles.)
It could well have been a maintenance situation. Those tend to be extremely tricky to do safely, in spite of all the precautions, especially if you want to do some diagnostics without taking the whole thing apart.
I have religious respect for those arms, if they're powered up you couldn't pay me to get within reach.
All of those are possible, but food handling usually uses less powerful robots (simply because they're a lot cheaper) so there may have been some complacency about them not being that dangerous.
(Physical barriers can be tricky to get right too, though - there have been incidents of "worker crushed between robot and inside of safety cage" though design standards have been updated in response to that.)
Readit News
Depending on the application there are different layers of safety surrounding these systems, including perimeter guards, optical barriers, limit switches, resistance based detection and so on. But when a system is broken someone has to go in and fix it, and you tend to do that with the robot powered up, some of the safety systems disabled so you can actually work on it and if you're really unlucky a motor will end up shorted against a + or - rail while you're within reach. This is obviously dangerous, and it is more dangerous because broken equipment can't be trusted to behave in a predictable way.
They won't stop. Not until whatever is obstructing has moved or the motor has burned out (or someone has the presence of mind to hit the e-stop). I've seen a 3" thick mount that must have weighed well over a ton sheared clear of its bolts (which themselves were an inch thick) by a malfunctioning servo on a very large lathe under construction (think 8 foot chuck for crane cable idler wheels). Do not fuck with servo systems unless you are 100% sure they are safe to approach or you may well end up dead or gravely injured.
My own process to ensure things were safe was to first run it in air without the spindle powered up, then on foam, chalk, and finally on aluminum in that order before progressing to steel.
And that saved my ass more than once.
All of that is to say that it seems like working on a manufacturing floor with robots has the potential to be so so dangerous. Getting into a rhythm and doing repetitive work all day, it’s easy to imagine slipping and ending up as human soup.
I thought OSHA and friends didn't allow this. Lockout/tagout is standard.
> some of the safety systems disabled
There's a simple one which ought to be more common: current limiter on the drive power supply. Makes everything slow and weak.
Yes it is, for plant workers it is a firing offense in many places.
But your typical maintenance tech that is supposed to work on the machinery itself rather than just to be using it may well use lockout/tagout to ensure that the machine is powered off when they are working on it and don't need to be able to do any kind of diagnostics. But short of a complete disassembly and testing each component in isolation - for which there often isn't really time and which given the pressure on maintenance technicians to get a piece of gear working again - people tend to take shortcuts such as to hook up analytics gear to a machine that is live. Stupid? Yes, absolutely. But this kind of stupidity is the result of usually many years of things working just fine and bad practices creep in. The guy that manages to get stuff done rapidly is the one that gets called out. And eventually an accident will happen. Weirdly enough even near accidents tend to reinforce the belief that it worked. When actually the lesson should be that it didn't but the person just got lucky.
One very memorable occasion at an industrial plant I visited is that someone got beaten up for removing a lock and I felt absolutely no sympathy for that guy. He could have well gotten someone killed (large 5 axis mill).
https://www.bloomberg.com/news/features/2017-03-23/inside-al...
The machine plunged into a large block of aluminum at full rapid speed with a 1 inch rougher end-mill. It then proceeded to run the cutting program, again, at full rapid rate. The sound it made is something that I will never forget. This thing was churning aluminum like it was butter. It was literally turning it into goo, not cutting it at all. It did not care one bit. The cutter was large enough and stubby enough that it did not break.
These things are fast enough that, by the time your shocked brain is able to respond it could have killed and dismembered you five times before you can even think of slamming the e-stop button.
The end-mill came out with a ball of aluminum, 2 inches in diameter, permanently welded to it. It was one of the scariest industrial accidents I've been around. Thankfully the enclosure and rigidity of the machine made it so that the only consequence was aluminum that looked like churned butter.
Years later I worked with an aerospace client who uses stir welding to assemble rockets. Same process, except you are doing it on purpose and with controls. The idea is that stir welding preserves the metallurgical properties of the material, whereas TIG/MIG welding might not preserve properties important to the design.
To echo your other comment, since that happened I learned to never trust CAM software or simulations, always run it on air first and, if necessary, machinable wax blocks. Decades later, I still don't consider myself a machinist at all. I am pretty decent at it. It's a complex art that requires time and dedication. I do too many other things to be that guy. I do have a VF-2 and a Bridgeport knee mill (full manual still) in my garage though...
I have a 30 Watt diode laser that came without any safety measures at all. It kind of surprised me that you could just fire it with a G-code and absolutely nothing stood between you and the beam. It cuts wood like butter though, up to 20 mm in a single pass (solid wood, not birch ply) and I'm super happy with it but that tool is going to go through some serious modifications to make it safe. It can set just about anything on fire fair distance away and will fry your eyeballs from half a kilometer if not more. Any idiot with a few bucks to spare can go and order one and they have absolutely no clue about the kind of damage a machine like that can do. It looks so innocent, it's completely deceptive.
Then we should work towards fixing broken robots using working robots.
I got hit right in the dick by a milling machine axis lever spinning up full throttle.
Only once, never again. Still surprised something terrible didn’t happen - hurt like a bitch. Shop dude says it happens pretty often though because people don’t look down at the caution tape telling you not to stand there.
https://m.yna.co.kr/amp/view/AKR20231108062151052
Looks like he was from the company of the said machine. The machine moves paprika boxes onto pallets.
https://www.theatlantic.com/technology/archive/2023/09/robot...
>One day in 1979, a robot at a Ford Motor Company casting plant malfunctioned—human workers determined that it was not going fast enough. And so 25-year-old Robert Williams was asked to climb into a storage rack to help move things along. The one-ton robot continued to work silently, smashing into Williams’s head and instantly killing him. This was reportedly the first incident in which a robot killed a human; many more would follow.
https://www.bbc.com/news/uk-england-london-64430454
- https://www.youtube.com/watch?v=8kIy-wEfUGU
- https://www.youtube.com/watch?v=qTMBlOyNlqc&t=442s
Imagine using one of those thinking "I'll be fine, it only eats the workers"
Deleted Comment
Looks like South Korea has an osha-equivalent https://en.wikipedia.org/wiki/Korea_Occupational_Safety_and_... but I have no idea if they have modern robot safety standards.
That was exactly my thought when reading it. If the e-stop had been depressed this likely would have never happened. But with the system live all bets are off, especially if it is already malfunctioning. You're essentially one queued G-code away from getting mauled.
I have religious respect for those arms, if they're powered up you couldn't pay me to get within reach.
(Physical barriers can be tricky to get right too, though - there have been incidents of "worker crushed between robot and inside of safety cage" though design standards have been updated in response to that.)