Over 20 years ago I had a Toshiba Tablet PC convertible that had a beam forming array of microphones and it came with software that let you point where you wanted to record from.
The use case was for lectures, you could tell the laptop to just record from behind it, pointing the beam in the direction of the professor.
In the golden age of mini camcorders, some Sony Handycams had "zoom" microphones which used beam forming to limit gathered sound to roughly the area equal to the what your sensor sees.
It's used widely in fancy videoconferencing setups.
The mic array for the room figures out who's talking and isolates the audio from them.
(Videoconferencing in large rooms has long picked the loudest microphone to use at any time, to avoid mixing in noise from other mics, but then the beamforming makes it that much better.)
I'm wondering if that's why those kinds of setups offer good audio if only one person speaks and there's clear pauses between people, but as soon as you have a quick back and forth or two people talking the audio turns into complete mush.
I wonder how that worked. Assuming the microphones were on the screen plane rather than the body, it wouldn't be able to tell the difference between "straight in front" and "straight behind".
Straight in front is likely to be unobstructed while straight behind is likely to be obstructed by computer hardware. Therefore, straight in front is likely to have crisp sound while straight behind is likely to be muffled and/or distorted by reflections.
As someone who made the mistake of putting a webcam cover over the tiny little microphone hole above the screen (it picks up very little besides impact noises now), it wouldn't be hard to have a mic hole facing in both directions to solve that problem
The attenuation provided by the tablet case / shell is quite significant. I bet they had some extra foam, or, something, to make it even stronger. So the "right behind" signal would be heard only if "right in front" is not readily drowning it.
Idea I've had for years but never got around to testing due to lack of compute:
Use a microphone array and LIDAR for ground truth, and train a diffusion model to "imagine" what the world looks like conditioned on some signal transformations of the microphone data only.
Could be used by autonomous vehicles to "see" pedestrians through bushes, early detect oncoming emergency vehicles, hear bicyclists before they are visible, and lots of other good things.
This already exists, it's the domain of inverse problems. Inverse problems consider a forward problem (in this case wave propagation) depending on some physical parameters or domain geometry, and deduce the parameters or geometry from observations.
Conceptually, it's quite simple, you need to derive a gradient of the output error with respect to the sought information. And then use that to minimize the error (= "loss function" or "objective" depending on field terminology), like you do in neural networks.
In many cases, the solution is not unique, unfortunately. The choice of emitters and receivers locations is crucial in the case you're interested in.
There's a lot of literature on this topic already, try "acoustic inverse problem" on google scholar.
I like it. I think you'd need to be in known motion around the area to build up a picture -- I don't think it would work with a microphone just sitting in place.
My (never finished) master's thesis was about something similar - taking advantage of the fact that (almost) all smartphones have at least two microphones I wanted to locate and separate a speaker in 3D.
A few takeaways:
-The sampling rate is slightly off between devices - approximately ±1 sample per second - not a lot, but you need to take that into account.
-Spectral characteristics in consumer microphones are all over the place - two phones of the same model, right out of the box, will have not only measurable, but also audible differences.
-Sound bounces off of everything, particularly concrete walls.
-A car is the closest thing to an anechoic chamber you can readily access.
-The Fourier transform of a Gaussian is a Gaussian, which is very helpful when you need to estimate the frequency of a harmonic signal (like speech) with a wavelength shorter than half your window, but just barely.
> - A car is the closest thing to an anechoic chamber you can readily access.
I recall a youtuber solving the anechoic chamber problem by finding a big empty field - nothing to reflect off of except the ground - and maybe putting some foam below the experiment.
It doesn't kill environmental noise, of course, but it apparently did a very good job of killing reflections from his own instruments.
In my case wind noise disturbed the signal too much. Normally there's additional processing which deals with it, but I was working with (next to) raw data.
I didn't have such a place at the time, but I found one and results weren't as good as in a car.
Sound deadening generally requires mass to work for lower frequencies and the seats absorbed them all nicely. I got some reflections from - I assume - the windows, but they were manageable in comparison. Didn't even produce much of a standing wave when I tried.
>The Fourier transform of a Gaussian is a Gaussian, which is very helpful when you need to estimate the frequency of a harmonic signal (like speech) with a wavelength shorter than half your window, but just barely.
I get the gaussian link. But, can you explain your point with more detail?
The log of a Gaussian is a parabola, which makes finding where exactly a peak in the spectrum lies a question of solving a quadratic equation.
My plan was to detect the frequency of the speaker by counting (with weights) which distance between peaks is the most common. I wanted to avoid calculating the power cepstrum as I felt that I was running out of computing power on the devices already[0] - a mistaken belief in the long run, but I was too proud of my little algorithm and how stable it was to let go.
[0] Sending raw sample data to a more powerful machine was out of the question as I wanted to remain within the bandwidth offered by Bluetooth at the time due to power consumption considerations.
Wow, this really puts into perspective how much work has to be put into even the most insignificant details of getting Linux to run on (Apple Silicon) Macs. I say "insignificant" with all due respect because, well, the built-in microphone sees very little use (except if you have forgotten your headset).
The built-in microphone is actually excellent, I often use it even when I have my AirPods Pro in because the sound quality is so much better
If you've got headphones with a wraparound microphone on its own arm then it could be better, but everyday headphones are limited by the position of the microphone
Yeah, no matter how good the microphone actually is on a headset, it uses an ancient codec so until we get Bluetooth 5.3 everywhere with lc3 codex then we won't actually have good mic input from headphones and headsets. I predict that this is all going to change this year and next year. But the full stack has to support it from headphones to Bluetooth chips to OS.
Everyday headphones are limited by the fact that people often use Bluetooth, and Bluetooth audio is just terrible tech that hasn't improved by much in the last 10 years, and still can't do more than 16kHZ when doing both input and output at the same time.
I think this isn't a problem if you're using Apple headphones with Apple devices, but anything else falls back to crappy BT quality, usually with some kind of terrible ANC to boot.
FOr me, crappy audio setups and apps trying to do too much audio processing are the primary reason of "Zoom fatigue". I've done a lot of calls over apps that transmit raw, high-quality audio with no processing whatsoever, and the experience is just so much better.
Plenty of good headsets do beamforming with their microphones as well, just depends on what you're running. Macbook mics are well above average, though, so I agree in most cases they'll be better unless you're picky about your headset mic quality.
The MBP mic is generally preferable to most headset boom mics in my experience with good noise reduction. You also get the benefit of not picking up extraneous mouth noises (gum chewing, coffee slurping, whatever)
I feel like 99% of people I conference with use regular headphones + MBP mic
Main problem with that setup is not being able to hear your own voice in the headphones (feedback, or whatever that's called) which can be annoying sometimes if using NC headphones
Actually my complaint relates to open office designs, the macbook mic picks up louder people from across the room. So if I do use headphones and the MBP mic, other people will hear random noise blurbs from anywhere in the office .
I don't think I recall having a meeting with anyone using plain headphones with the laptop mic instead of a headset of some kind. Wired headphones without a mic are somewhat unusual nowadays to begin with outside audio file circles.
AirPods of various versions is common, as is many other buds. Enterprise headsets like those from EPOS (the old Sennheiser Communication) and Jabra (with or without boom) and speakerphones are common in corporate settings, casual headsets (e.g., Sony, Bose) and wired gaming headsets are common at home.
Well it is simple if you use the whole package as delivered (although Apple has been straying off the road it paved for quite a while now).
The point is, everything they make is vertically integrated. They want to deliver a feature (like Airdrop or Continuity), they will cut across the stack to get it done. If you go the DIY route (which is what Asahi is effectively all about), you get to also DIY the missing software pieces.
The upside is that the entire ecosystem gets to benefit from that work (see e.g. the new DSP in PipeWire). PC hardware is generally crap, and so is Apple's if you omit these extra bits. But "the whole package" sets the bar quite a bit higher. I want to see the FOSS ecosystem meet that bar.
The three-mic array is also found in Intel-based Retina MacBooks, so this might also be useful for proper audio support on that older hardware. (Some early Retina MacBook Pros have a two-mic array only, but most have the full three-mic array.)
Because most mics are still using Bluetooth 5.0 I use the microphone on my Mac even when I'm wearing a headset. Otherwise, it puts me into a weird codec mode of ancient history where I get downgraded to a low bit rate and even my audio input to my ears sounds horrible then. So I always use the Mac microphone when possible.
It's more annoying on Linux where you have to manually switch... at least most apps in windows/mac will automagically put my headset in the correct mode.
I always set my microphone to MacBook's even when wearing a headphones, because the quality is incredibly good even in noisy environments. In Zoom I also set "Original sound for musicians" on if in a quiet location. So much more natural sound.
You can get surprisingly good results from cheap laptop hardware (as well as fancier hardware like an MBP) using software DSP techniques. One of the things I'm pleased about is that quite a bit of Asahi's audio work is just as applicable to generic laptops as it is to Macs.
I already use the Bankstown bass harmonics synthesis plugin developed for Asahi and a convolution EQ on a cheap HP laptop, with startlingly impressive results, using the Pipewire plugin chain autoload feature also developed for Asahi.
I suspect there are quite a few use cases for this beamformer outside of the Asahi ecosystem as well.
Regarding the SIMD optimizations, the authors may want to look into faer. I haven't had a great experience with its underlying library pulp, as I'm trying to things that go beyond its linear algebra roots, but if the goal is primarily to accelerate linear algebra operations, I think it will go well.
I've got a blog post and associated podcast on Rust SIMD in the pipeline, we'll touch on this.
> the microphone array found in the following Apple Silicon laptops:
> MacBook Pro 13" (M1/M2)
> MacBook Air 13" (M1/M2)
> MacBook Pro 14" (M1 Pro/Max, M2 Pro/Max)
> MacBook Pro 16" (M1 Pro/Max, M2 Pro/Max)
> MacBook Air 15" (M2)
Does it mean M2/M3 don't have similar array of microphones or rather not tested?
I'm even curious if this is only supported on Linux or MacOS as well - not sure if apple provides dedicated microphone stream for each mic?
Readit News
The use case was for lectures, you could tell the laptop to just record from behind it, pointing the beam in the direction of the professor.
Amazing idea and something I haven't seen since.
Another great idea.
Oh. They still make similar stuff: https://electronics.sony.com/imaging/imaging-accessories/all...
Deleted Comment
The mic array for the room figures out who's talking and isolates the audio from them.
(Videoconferencing in large rooms has long picked the loudest microphone to use at any time, to avoid mixing in noise from other mics, but then the beamforming makes it that much better.)
https://www.sennheiser.com/en-us/catalog/products/meeting-an...
Use a microphone array and LIDAR for ground truth, and train a diffusion model to "imagine" what the world looks like conditioned on some signal transformations of the microphone data only.
Could be used by autonomous vehicles to "see" pedestrians through bushes, early detect oncoming emergency vehicles, hear bicyclists before they are visible, and lots of other good things.
Conceptually, it's quite simple, you need to derive a gradient of the output error with respect to the sought information. And then use that to minimize the error (= "loss function" or "objective" depending on field terminology), like you do in neural networks.
In many cases, the solution is not unique, unfortunately. The choice of emitters and receivers locations is crucial in the case you're interested in.
There's a lot of literature on this topic already, try "acoustic inverse problem" on google scholar.
I like it. I think you'd need to be in known motion around the area to build up a picture -- I don't think it would work with a microphone just sitting in place.
A few takeaways:
-The sampling rate is slightly off between devices - approximately ±1 sample per second - not a lot, but you need to take that into account.
-Spectral characteristics in consumer microphones are all over the place - two phones of the same model, right out of the box, will have not only measurable, but also audible differences.
-Sound bounces off of everything, particularly concrete walls.
-A car is the closest thing to an anechoic chamber you can readily access.
-The Fourier transform of a Gaussian is a Gaussian, which is very helpful when you need to estimate the frequency of a harmonic signal (like speech) with a wavelength shorter than half your window, but just barely.
I recall a youtuber solving the anechoic chamber problem by finding a big empty field - nothing to reflect off of except the ground - and maybe putting some foam below the experiment.
It doesn't kill environmental noise, of course, but it apparently did a very good job of killing reflections from his own instruments.
Sound deadening generally requires mass to work for lower frequencies and the seats absorbed them all nicely. I got some reflections from - I assume - the windows, but they were manageable in comparison. Didn't even produce much of a standing wave when I tried.
I get the gaussian link. But, can you explain your point with more detail?
My plan was to detect the frequency of the speaker by counting (with weights) which distance between peaks is the most common. I wanted to avoid calculating the power cepstrum as I felt that I was running out of computing power on the devices already[0] - a mistaken belief in the long run, but I was too proud of my little algorithm and how stable it was to let go.
[0] Sending raw sample data to a more powerful machine was out of the question as I wanted to remain within the bandwidth offered by Bluetooth at the time due to power consumption considerations.
Or, to quote the progress report (https://asahilinux.org/2025/03/progress-report-6-14/#is-this...): "This is Apple though. Nothing is ever simple."
If you've got headphones with a wraparound microphone on its own arm then it could be better, but everyday headphones are limited by the position of the microphone
I think this isn't a problem if you're using Apple headphones with Apple devices, but anything else falls back to crappy BT quality, usually with some kind of terrible ANC to boot.
FOr me, crappy audio setups and apps trying to do too much audio processing are the primary reason of "Zoom fatigue". I've done a lot of calls over apps that transmit raw, high-quality audio with no processing whatsoever, and the experience is just so much better.
The MBP mic is generally preferable to most headset boom mics in my experience with good noise reduction. You also get the benefit of not picking up extraneous mouth noises (gum chewing, coffee slurping, whatever)
I feel like 99% of people I conference with use regular headphones + MBP mic
Main problem with that setup is not being able to hear your own voice in the headphones (feedback, or whatever that's called) which can be annoying sometimes if using NC headphones
Monitoring.
There are umpteenth ways to do that, and I find headsets themselves do it the most poorly of all (if they have the feature at all).
> The MBP mic is generally preferable to most headset boom mics
Another benefit is not paying the '90s GSM handsfree BT profile codec pain (at the cost of A2DP having slightly higher latency)
AirPods of various versions is common, as is many other buds. Enterprise headsets like those from EPOS (the old Sennheiser Communication) and Jabra (with or without boom) and speakerphones are common in corporate settings, casual headsets (e.g., Sony, Bose) and wired gaming headsets are common at home.
The point is, everything they make is vertically integrated. They want to deliver a feature (like Airdrop or Continuity), they will cut across the stack to get it done. If you go the DIY route (which is what Asahi is effectively all about), you get to also DIY the missing software pieces.
The upside is that the entire ecosystem gets to benefit from that work (see e.g. the new DSP in PipeWire). PC hardware is generally crap, and so is Apple's if you omit these extra bits. But "the whole package" sets the bar quite a bit higher. I want to see the FOSS ecosystem meet that bar.
I already use the Bankstown bass harmonics synthesis plugin developed for Asahi and a convolution EQ on a cheap HP laptop, with startlingly impressive results, using the Pipewire plugin chain autoload feature also developed for Asahi.
I suspect there are quite a few use cases for this beamformer outside of the Asahi ecosystem as well.
I've got a blog post and associated podcast on Rust SIMD in the pipeline, we'll touch on this.
[1]: https://docs.rs/faer/latest/faer/
Does it mean M2/M3 don't have similar array of microphones or rather not tested?
I'm even curious if this is only supported on Linux or MacOS as well - not sure if apple provides dedicated microphone stream for each mic?
MacOS has its own software deep within the system for handling this; it's only exposed as a normal microphone to application software.