> Oh yes, there are some old computers that won't boot from USB
Back in the day, there were some old computers that wouldn't boot from a CD drive. I recall that the solution back then was very simple: within the ISO image, there was a bootable floppy image, which you could write to a normal floppy disk (actualy, there were IIRC usually two such images, one for 1.44 floppy drives, and another for the older 1.2 floppy drives). You would boot and load the kernel from that floppy, but the installer would look for the rest of the files on the CD drive.
The same idea could be used here: for those old computers which have a BIOS which do not understand how to boot from a USB drive, a small bootable CD image containing only the bootloader and the kernel could be made available, either within the USB image or as a separate download.
For a while, in 1994, I had some weird SCSI card (soundblaster with SCSI, I think) that couldn't boot Linux off a harddrive. (It probably could, I just couldn't get it to work...) So I just left a floppy with a kernel in the drive. The system was generally running 24/7, but I wanted it there in case it did reboot...
ISO 9660 discs still have bootable floppy images in them. The El Torito standard for bootable CDs specifies that a floppy image be incorporated into the disc image. The PC BIOS knows how to find this image, load it into memory, and boot it as a virtual floppy.
But of course, ugly hacks like this were only ever a peecee problem. Macs could boot CD-ROMs like any other drive for as long as they've had CD-ROMs.
> But of course, ugly hacks like this were only ever a peecee problem. Macs could boot CD-ROMs like any other drive for as long as they've had CD-ROMs.
This is a bit ridiculous, since in order to be bootable, a Mac requires the CD-ROM to contain a HFS filesystem.
PCs could also boot "like any other drive" from CD-ROM if they forced the CD-ROMs to contain a MBR.
Actually, that is exactly what El Torito is: putting a boot record on a CDROM, so that it can boot "like any other drive". The fact that the boot record is most commonly found in an image of a floppy disk is an implementation detail (you can have whatever you want, it's just that floppy is easier to work with), and hardly "a hack".
"A hack" would be to force the CD-ROM to contain a hybridized HFS filesystem, despite the entire raison d'être for ISO being to avoid having to put multiple filesystems in CDs.
Somewhat related… for a while now Microsoft have provided publicly downloadable iso images for Windows 10 that you can’t image to a usb drive without needing special EFI NTFS software from a third party which in turn requires disabling secure boot (as it’s not secure boot signed)
Why you may ask? Simply because the iso contains a file larger than 4GB (install.wim) so you cannot use a FAT32 formatted usb drive (FAT32 is needed for EFI boot). Instead tools like Rufus provide their own EFI NTFS boot loader.
Interestingly Microsoft’s own usb creator doesn’t include install.wim but instead install.esd which is less than 4GB so can use FAT32. I have no idea why the iso uses a wim file whereas the windows usb creator uses an esd file. I do know the esd version is a little slower to install (only a minute or two). I guess because of higher compression.
This is a pain because making a Windows bootable installer on Linux or macOS is now more complicated than it need be as you cannot simply put the iso onto the usb drive with dd or gdisk as you would normally do.
Does anyone know why this is the case? As a workaround you can use Microsoft’s usb media creator to build an iso with the esd file then use dd as usual. But of course this requires a Windows system to run the Windows only usb creation tool and is a slower process than just downloading an iso directly. Annoying.
On Linux look at wimsplit which will convert the .wim into a series of .swm of smaller size.
As for why my guess is the .iso came from the standard distribution image used in CDs and handed to OEMs and whatnot. ESD doesn't allow direct image modification/slipstreaming and the official DVDs don't care if the file is >4GB as long as it fits on a disc.
The Windows USB tool actually creates an ISO based on the current update files on the Windows Update servers (for the major distribution, I don't think it includes minor updates), it's not pulling down a precompiled .iso. You'll actually get a different hash whenever you make this ISO as a result. Similarly if you dump to a USB drive it's just pulling the files to put there not extracting a .iso containing an .esd.
As to why they haven't implemented a better way for Linux/Mac users without access to a Windows machine to create a bootable Windows USB? It probably isn't a common enough problem for them to care.
EDIT: I just realized no of the bellow helps you if you are not on Windows =(
The documented way to work around the 4GB limitation is to use the dism.exe /Split-Image command. This will turn the one large .wim file into multiple smaller files.
I ran into this when I was working on SimpleDiskImager. It is my disk image writer that tries to find a balance between dd's literal-minded copying of bytes (which does not correctly handle GPT partitioning) and the opinionated and invasive modifications that Rufus makes (not yet compatible with secure boot). I have not yet integrated support for writing Windows installer ISOs yet, but you can see the start of it in the ConvertEfiIsoToFat directory.
Yeah I noticed this. I tried using DD to 'burn' the windows 11 ISO to a USB stick and it didn't work. I had to use Rufus as per MS' instructions. It's very weird to me that they include a third-party app in this workflow.
I've had success with using exFAT on my Mac for installing Windows, but I think that support for this in EFI on most hardware may be even less common than NTFS support in EFI.
Using DD for windows images has been a pain for years, even before windows 10, manually copying the image content with rsync to a correctly formatted external drive for me was the most reliable way in the Win8.1 era.
Modern PCs made in the last 5+ years or so come with EFI drivers to read NTFS partitions. FAT32 is not "needed for EFI boot". You can extract the ISO (e.g. with 7z) onto a USB stick and you're done in most cases.
The spec allows it but whether FAT32 needed depends solely on the board not the age. I've got plenty of recent boards that don't take NTFS/exFAT - I usually find out when I'm scratching my head as to why it doesn't show up in the boot menu.
Out of interest I did a little test this afternoon.
Got a 128GB SanDisk USB flash drive. Used diskpart to clean the disk then created a single NTFS GPT partition. Extracted the current Windows 11 ISO they offer for download onto the drive using 7-Zip.
Tried to boot on a Surface Studio Laptop, a Lenovo ThinkPad X1 Carbon from 2019 and an Alienware M15 from this year. Failed on all three.
Just to be on safe side I used bootsect to ensure the USB is bootable but again failed to boot on all three.
I then used the MS Media Creation Tool which happily creates a FAT32 formatted USB drive. This naturally boots fine on all systems.
So while technically me saying FAT32 is needed for EFI is not correct the reality is FAT32 is what you can be [almost?] certain is supported on any system using EFI.
Not even on Microsoft's brand new flagship laptop or a brand new, high end gaming laptop from Dell appear to have NTFS support in their EFI.
I wish it were as simple as just extract the iso to an NTFS USB drive and boot.
Some systems do indeed come with NTFS EFI support. However it isn’t a common thing.
For example the new Microsoft Surface Studio Laptop I was using a few weeks ago does not support NTFS and the support documentation gives you instructions to create a FAT32 partition. This is why it’s fresh in my mind.
I haven't touched a physical optical drive in many years, but I still use ISO's regularly for building virtual machine images or booting servers into something quickly. I can get away with plain pxe/netboot for distro's I'm familiar with the packaging of but an ISO is the distro packaging unit I know that will work wherever it is needed.
EasyOS might be targeted elsewhere? Compared to say debian/ubuntu/rhel
That's an interesting point. It's easy to mount an ISO in VirtualBox, but I don't see a way to mount any sort of "virtual USB flash drive". I wonder what's the easiest[1] way to install an OS in a VM if you don't have an ISO.
Even more interesting is EasyOS's official install instructions[2] seem to tell you to download a live CD ISO for an old version, and move from there to the latest version. I guess they're not completely free of needing ISOs just yet.
[1] Obviously excluding things like virt-install. I'm talking about live booting an interactive OS or installer.
> I wonder what's the easiest[1] way to install an OS in a VM if you don't have an ISO
I can't recall seeing any generic disk image alternatives to ISO's. It's generally download a virtual disk image (e.g Vagrant), and if the image is not in the right format convert it before attaching.
The only other work flow I use is to netboot with iPXE [0]. All it needs is a http(s) endpoint for the initrd/kernel and away you go, but there is a bit of investment in setting up the initial boot environment.
I ran headfirst into this "non-writeable" business a few years back. For what I was doing, though, I went straight into the virtualization thing, which seems awful popular with the kids these days.
Forensics still needs the stick, though, although I am sure one of my grandkids knows some super secret trick that does the same thing in half the time/money.
Also, add me to the queue of people who were jazzed about an article detailing someone walking away from ISO specifications. I've lost count of the ISO "specifications" that turned out to be nothing more than freeways into the garage of a sole-source vendor who "maintains" some archaic file specification that apparently I can't do without. And who runs the Working Group. Fancy that! What a co-inky-dink.
And AS9100 governance has gone from "middling poor" to "cannonball dive into the thunder pot" during the course of my professional career, now flirting with "outright extortion" . Now, I do realize AS9100 is SAE and not ISO proper, but a ton of the bad stuff is coming straight from old ISO hands either in their original chairs or having moved around to SAE or IAQG.
I have to say the reason I still want an iso is at least I have a much higher guarantee it will work. I just spent about 4 hours this week trying to get clonezilla to boot from usb and could not get it to work. EFi. Legacy. Everything I could think of. And this was on a reasonably modern server (2015/16 ish lenovo with a e3-1270 V5 )
Come to find out it was the fact that it was a usb 3.0 flash drive. Grabbed a 2.0 drive and everything worked fine.
Had I had a dvd burner (Ironically the computer in question did have one but it was not useable till i fixed it.) I would have saved myself significant amounts of time. (Or randomly grabbed a crappier flash drive.)
Yes, I've found USB 3 and above to be significantly less compatible and reliable than 2.0 and below. I can always trust that whatever I have plugged into a 2.0 port will work the moment the PC comes on, be it a flash drive or a DVD drive or a keyboard/mouse. That isn't always the case with a 3.0 port.
The reason for the problems like you describe is that on many motherboards ports with later USB versions were implemented with discrete USB host controllers or ever hubs, while ports of older standards were integrated into the southbridge. This is one of the reasons why the first thing I do encountering a previously unexplored PC is looking on motherboard block diagram in the motherboard's manual to understand how ports are routed.
I have the opposite experience wrt. ISOs and plain (dd) images. If I burn an ISO to a USB stick, I can use the whole stick. But if I burn a dd image to the stick, the size must match exactly, otherwise I have to futz around with fdisk and resize2fs. Sometimes a 4GB stick has less memory than another 4GB stick, so you can't restore.
And when creating images, you have to take extra care to zero out unused space, otherwise your dd image will not be compressible.
Frankly I just want a small file that I can right-click or drag into Rufus and burn my USB stick or SD card ASAP. I used to know exactly what was going on when making a bootable floppy; the file containing the bootloader had to be in certain sectors, so you had to create it with FORMAT, but all other files could just be copied on. I don't know if any special sector layout is neccessary nowadays, I think for EFI boot it just requires the EFI directory. And tools like Rufus probably just copy file by file to the USB drive, not sector wise.
For EFI it's just the directory and probably using FAT32. And if there's a partition table it should be marked the right type. But there are no magic files or sectors.
> Despite very careful instruction, and even writing down simple steps to make a call, etc., I could see her eyes glazing over. She just didn't want to know, and after awhile I had to buy her another keyboard flip-phone.
I feel like I'm already getting there with some things and I'm half that age. If something as disruptive as systemd or GNOME 3 came out again, I think I wouldn't even listen to the rationales and just follow the old fork this time around. Things work as they are for me.
Also the change has to bring some added value. When you get older time becomes scarce and you don't have time to test every shiny new toy and fix its possible issues.
For those that don't even want to download and dd to a usb stick any more (or only do it once!), check out https://netboot.xyz/. I've actually gone ahead and flashed it over my network card's stock PXE booter, and so now booting into a live image to fix something is completely trivial.
Every time there’s a conversation about ISO’s and booting someone mentions netboot, I still don’t understand at a granular level how it works and the native documentation, at least to me, appears to be written in Greek. I can’t make heads nor tails of it. Do you have a guide you followed that explained how it worked and what to do?
It basically bundles iPXE which allows one to boot over the network including ethernet and I think WiFi although WiFi depends on drivers. It downloads an bootable image and chain boots into it.
We used in our ML data center to iterate on our NixOS images meaning we could push out updates to the underlying OS and then just trigger a restart via IPMI and make it download the new images.
Note that we didn't end up using netboot at the end, we just use iPXE directly.
PC Network booting these days is usually PXE booting.
You can do it with BIOS or UEFI styles, but I've only had success with the BIOS version. I setup my PXE boot setup over a long time, so I don't have a guide, but here's my idea of the steps the boot process takes.
a) (if enabled and selected for booting) the system firmware jumps to the Option ROM on your NIC to boot. These days, almost all cards have a PXE rom, but if you have an ancient card, it probably has a socket fot a boot rom. Getting that setup is possible but outside my scope.
b) the boot rom includes a driver for the NIC and usually Intel's PXE software; either way it will do a DHCP (or sometimes BOOTP) request and ask for a 'next server' and indicate the platform (x86 bios, x86 uefi, amd64 bios, amd64 uefi, sparc variations, etc, etc). The DHCP server will assign an IP and all that junk, but also provide a pointer to the next server (and a path), used to fetch more code to run.
c) next server is traditionally a TFTP server, and the path may be client or platform specific. More modern pxe stacks can do http(s), usually at higher performance
d) I find it easiest to have next server point at pxelinux, part of syslinux that offers a nice menu system and http(s) support that's usually a lot faster than tftp
e) from the menu, you can often selection options that will load a kernel and modules and boot with a command line; in the boot process, usually all the pxe stuff is terminated or discarded.
f) You can also load disk images (floppy, hard drive, iso) and boot from them using memdisk from syslinux. This comes with some caveats; MEMDISK is BIOS only, it hooks the (defacto) standard BIOS disk calls to provide access and doesn't have a UEFI equivalent (AFAIK, corrections welcome), but also if you use an OS that doesn't use BIOS disk access (most protected mode OSes), it needs to support MEMDISK or you can't access the images. This means only some Linux images work, although recent images often do. I've made a patch for FreeBSD, but it's been in review for a long time. Older FreeBSD setup images worked because the setup filesystem was a disk image loaded from the disc by the bootloader, you'd need enough memory to have the cd disc image loaded by memdisk and the setup filesystem loaded by the bootloader and enough left to run the installer, but it's not too bad on today's machines.
So, TLDR, you need to fiddle with your dhcp server, setup a tftp server, copy some files for a bootmenu and/or something to boot. And finally, tweak the boot settings on your netbooting device.
Useful things I've done with this include booting Debian installers, using weird 3rd party software to install windows XP, booting FreeBSD installers with my patch, running disk firmware updates packaged as linux isos (intel), and most fun, running my hobby OS without having to setup a bootdisk (it's multiboot compatible, so pxelinux can boot it easily)
Readit News
Back in the day, there were some old computers that wouldn't boot from a CD drive. I recall that the solution back then was very simple: within the ISO image, there was a bootable floppy image, which you could write to a normal floppy disk (actualy, there were IIRC usually two such images, one for 1.44 floppy drives, and another for the older 1.2 floppy drives). You would boot and load the kernel from that floppy, but the installer would look for the rest of the files on the CD drive.
The same idea could be used here: for those old computers which have a BIOS which do not understand how to boot from a USB drive, a small bootable CD image containing only the bootloader and the kernel could be made available, either within the USB image or as a separate download.
https://www.plop.at/en/bootmanager/download.html
I can't help but imagine it as working similiar to the Floppotron...
https://www.youtube.com/watch?v=oGfkPCZYfFw
But of course, ugly hacks like this were only ever a peecee problem. Macs could boot CD-ROMs like any other drive for as long as they've had CD-ROMs.
This is a bit ridiculous, since in order to be bootable, a Mac requires the CD-ROM to contain a HFS filesystem. PCs could also boot "like any other drive" from CD-ROM if they forced the CD-ROMs to contain a MBR.
Actually, that is exactly what El Torito is: putting a boot record on a CDROM, so that it can boot "like any other drive". The fact that the boot record is most commonly found in an image of a floppy disk is an implementation detail (you can have whatever you want, it's just that floppy is easier to work with), and hardly "a hack".
"A hack" would be to force the CD-ROM to contain a hybridized HFS filesystem, despite the entire raison d'être for ISO being to avoid having to put multiple filesystems in CDs.
Why you may ask? Simply because the iso contains a file larger than 4GB (install.wim) so you cannot use a FAT32 formatted usb drive (FAT32 is needed for EFI boot). Instead tools like Rufus provide their own EFI NTFS boot loader.
Interestingly Microsoft’s own usb creator doesn’t include install.wim but instead install.esd which is less than 4GB so can use FAT32. I have no idea why the iso uses a wim file whereas the windows usb creator uses an esd file. I do know the esd version is a little slower to install (only a minute or two). I guess because of higher compression.
This is a pain because making a Windows bootable installer on Linux or macOS is now more complicated than it need be as you cannot simply put the iso onto the usb drive with dd or gdisk as you would normally do.
Does anyone know why this is the case? As a workaround you can use Microsoft’s usb media creator to build an iso with the esd file then use dd as usual. But of course this requires a Windows system to run the Windows only usb creation tool and is a slower process than just downloading an iso directly. Annoying.
As for why my guess is the .iso came from the standard distribution image used in CDs and handed to OEMs and whatnot. ESD doesn't allow direct image modification/slipstreaming and the official DVDs don't care if the file is >4GB as long as it fits on a disc.
The Windows USB tool actually creates an ISO based on the current update files on the Windows Update servers (for the major distribution, I don't think it includes minor updates), it's not pulling down a precompiled .iso. You'll actually get a different hash whenever you make this ISO as a result. Similarly if you dump to a USB drive it's just pulling the files to put there not extracting a .iso containing an .esd.
As to why they haven't implemented a better way for Linux/Mac users without access to a Windows machine to create a bootable Windows USB? It probably isn't a common enough problem for them to care.
https://github.com/ventoy/Ventoy
The documented way to work around the 4GB limitation is to use the dism.exe /Split-Image command. This will turn the one large .wim file into multiple smaller files.
https://docs.microsoft.com/windows-hardware/manufacture/desk...
I ran into this when I was working on SimpleDiskImager. It is my disk image writer that tries to find a balance between dd's literal-minded copying of bytes (which does not correctly handle GPT partitioning) and the opinionated and invasive modifications that Rufus makes (not yet compatible with secure boot). I have not yet integrated support for writing Windows installer ISOs yet, but you can see the start of it in the ConvertEfiIsoToFat directory.
https://github.com/AustinWise/SimpleDiskImager
Using DD for windows images has been a pain for years, even before windows 10, manually copying the image content with rsync to a correctly formatted external drive for me was the most reliable way in the Win8.1 era.
Deleted Comment
Got a 128GB SanDisk USB flash drive. Used diskpart to clean the disk then created a single NTFS GPT partition. Extracted the current Windows 11 ISO they offer for download onto the drive using 7-Zip.
Tried to boot on a Surface Studio Laptop, a Lenovo ThinkPad X1 Carbon from 2019 and an Alienware M15 from this year. Failed on all three.
Just to be on safe side I used bootsect to ensure the USB is bootable but again failed to boot on all three.
I then used the MS Media Creation Tool which happily creates a FAT32 formatted USB drive. This naturally boots fine on all systems.
So while technically me saying FAT32 is needed for EFI is not correct the reality is FAT32 is what you can be [almost?] certain is supported on any system using EFI.
Not even on Microsoft's brand new flagship laptop or a brand new, high end gaming laptop from Dell appear to have NTFS support in their EFI.
I wish it were as simple as just extract the iso to an NTFS USB drive and boot.
For example the new Microsoft Surface Studio Laptop I was using a few weeks ago does not support NTFS and the support documentation gives you instructions to create a FAT32 partition. This is why it’s fresh in my mind.
Not a big deal, just annoying.
EasyOS might be targeted elsewhere? Compared to say debian/ubuntu/rhel
Even more interesting is EasyOS's official install instructions[2] seem to tell you to download a live CD ISO for an old version, and move from there to the latest version. I guess they're not completely free of needing ISOs just yet.
[1] Obviously excluding things like virt-install. I'm talking about live booting an interactive OS or installer.
[2] https://easyos.org/install/easy-frugal-installation.html, search for "Easy live-CD"
I can't recall seeing any generic disk image alternatives to ISO's. It's generally download a virtual disk image (e.g Vagrant), and if the image is not in the right format convert it before attaching.
The only other work flow I use is to netboot with iPXE [0]. All it needs is a http(s) endpoint for the initrd/kernel and away you go, but there is a bit of investment in setting up the initial boot environment.
[0]: https://ipxe.org/
Forensics still needs the stick, though, although I am sure one of my grandkids knows some super secret trick that does the same thing in half the time/money.
Also, add me to the queue of people who were jazzed about an article detailing someone walking away from ISO specifications. I've lost count of the ISO "specifications" that turned out to be nothing more than freeways into the garage of a sole-source vendor who "maintains" some archaic file specification that apparently I can't do without. And who runs the Working Group. Fancy that! What a co-inky-dink.
And AS9100 governance has gone from "middling poor" to "cannonball dive into the thunder pot" during the course of my professional career, now flirting with "outright extortion" . Now, I do realize AS9100 is SAE and not ISO proper, but a ton of the bad stuff is coming straight from old ISO hands either in their original chairs or having moved around to SAE or IAQG.
Come to find out it was the fact that it was a usb 3.0 flash drive. Grabbed a 2.0 drive and everything worked fine.
Had I had a dvd burner (Ironically the computer in question did have one but it was not useable till i fixed it.) I would have saved myself significant amounts of time. (Or randomly grabbed a crappier flash drive.)
And when creating images, you have to take extra care to zero out unused space, otherwise your dd image will not be compressible.
Frankly I just want a small file that I can right-click or drag into Rufus and burn my USB stick or SD card ASAP. I used to know exactly what was going on when making a bootable floppy; the file containing the bootloader had to be in certain sectors, so you had to create it with FORMAT, but all other files could just be copied on. I don't know if any special sector layout is neccessary nowadays, I think for EFI boot it just requires the EFI directory. And tools like Rufus probably just copy file by file to the USB drive, not sector wise.
I feel like I'm already getting there with some things and I'm half that age. If something as disruptive as systemd or GNOME 3 came out again, I think I wouldn't even listen to the rationales and just follow the old fork this time around. Things work as they are for me.
We used in our ML data center to iterate on our NixOS images meaning we could push out updates to the underlying OS and then just trigger a restart via IPMI and make it download the new images.
Note that we didn't end up using netboot at the end, we just use iPXE directly.
You can do it with BIOS or UEFI styles, but I've only had success with the BIOS version. I setup my PXE boot setup over a long time, so I don't have a guide, but here's my idea of the steps the boot process takes.
a) (if enabled and selected for booting) the system firmware jumps to the Option ROM on your NIC to boot. These days, almost all cards have a PXE rom, but if you have an ancient card, it probably has a socket fot a boot rom. Getting that setup is possible but outside my scope.
b) the boot rom includes a driver for the NIC and usually Intel's PXE software; either way it will do a DHCP (or sometimes BOOTP) request and ask for a 'next server' and indicate the platform (x86 bios, x86 uefi, amd64 bios, amd64 uefi, sparc variations, etc, etc). The DHCP server will assign an IP and all that junk, but also provide a pointer to the next server (and a path), used to fetch more code to run.
c) next server is traditionally a TFTP server, and the path may be client or platform specific. More modern pxe stacks can do http(s), usually at higher performance
d) I find it easiest to have next server point at pxelinux, part of syslinux that offers a nice menu system and http(s) support that's usually a lot faster than tftp
e) from the menu, you can often selection options that will load a kernel and modules and boot with a command line; in the boot process, usually all the pxe stuff is terminated or discarded.
f) You can also load disk images (floppy, hard drive, iso) and boot from them using memdisk from syslinux. This comes with some caveats; MEMDISK is BIOS only, it hooks the (defacto) standard BIOS disk calls to provide access and doesn't have a UEFI equivalent (AFAIK, corrections welcome), but also if you use an OS that doesn't use BIOS disk access (most protected mode OSes), it needs to support MEMDISK or you can't access the images. This means only some Linux images work, although recent images often do. I've made a patch for FreeBSD, but it's been in review for a long time. Older FreeBSD setup images worked because the setup filesystem was a disk image loaded from the disc by the bootloader, you'd need enough memory to have the cd disc image loaded by memdisk and the setup filesystem loaded by the bootloader and enough left to run the installer, but it's not too bad on today's machines.
So, TLDR, you need to fiddle with your dhcp server, setup a tftp server, copy some files for a bootmenu and/or something to boot. And finally, tweak the boot settings on your netbooting device.
Useful things I've done with this include booting Debian installers, using weird 3rd party software to install windows XP, booting FreeBSD installers with my patch, running disk firmware updates packaged as linux isos (intel), and most fun, running my hobby OS without having to setup a bootdisk (it's multiboot compatible, so pxelinux can boot it easily)