Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,24 @@ section here does not ship.
Releases before 0.3.0 are on the
[Releases page](https://github.com/TerrifiedBug/yap/releases).

## 0.3.1

Meeting detection now names what has your microphone, and ignoring something
actually makes it go away.

- The prompt and the recording banner name the client in the two cases where
they used to say nothing at all. A helper process counts as the app that
owns it, so one "Ignore" covers every helper Chrome or Teams starts. A
background process counts as itself, so the speech service macOS keeps
listening with when "Hey Siri" is on shows up as `CoreSpeech` and can be
ignored like an app. Before, either of those produced a bare "Recording
meeting" pill with no Ignore button on it — nothing to click, and no way to
stop it happening again short of turning detection off.
- An ignored app is now genuinely out of the picture. It used to stay the
client yap was following, which hid everything else behind it: once you had
ignored a process that holds the microphone for hours, no real call could be
detected until it let go.

## 0.3.0

yap is an app now. Everything that used to need a terminal happens in the menu
Expand Down
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,13 @@ and name, and the + and − under it add one ahead of time or stop ignoring the
one you select. Detection stays fail-open — an app you have never excluded still
gets offered, even one yap has never heard of.

What holds the microphone is not always an app you launched. A helper process
belonging to one counts as its app — Ignore covers every helper Chrome or Teams
starts — and a background daemon counts as itself: with "Hey Siri" on, macOS's
speech service keeps an input stream open, so detection reads it as a call and
offers to ignore it under the name `CoreSpeech`. Ignore it once and it stops
being a meeting.

`mic_voice_processing` cancels speaker echo on the mic track. On by default: a
call coming out of your speakers goes back into the mic. Without it, the other
side gets transcribed twice, the second time as you. If some audio route
Expand Down
36 changes: 18 additions & 18 deletions Sources/yap/Daemon.swift
Original file line number Diff line number Diff line change
Expand Up @@ -661,24 +661,23 @@ final class Daemon: NSObject, NSApplicationDelegate {
/// call: a manual session holds the mic, and detection must stay down
/// until it ends.
private func wire(_ detector: MeetingDetector) {
detector.onMeetingStart = { [weak self, weak detector] pid, appName in
detector.onMeetingStart = { [weak self, weak detector] pid, app in
guard let self else { return }

// Before anything else, including the AX title lookup and the
// back-to-back stop: an excluded app is invisible to every piece
// of meeting logic, not merely unprompted. Read fresh at each
// event, the same standing-consent pattern as auto-record below.
let bundleID = MeetingTitle.bundleID(forPID: pid)
if let bundleID, Config.meetingExcludedApps().contains(bundleID) {
warn("◇ \(appName ?? bundleID) is excluded — ignoring")
// Marks the pid, so the detector stops re-firing every poll;
// its end-of-meeting path clears the mark on its own.
detector?.declineCurrentMeeting()
if let app, Config.meetingExcludedApps().contains(app.bundleID) {
warn("◇ \(app.name) is excluded — ignoring")
// Drops the client out of the detector's scan, so it neither
// re-fires every poll nor hides another app behind it.
detector?.ignoreCurrentClient()
return
}

let title = MeetingTitle.capture(forCapturePID: pid)
let who = appName ?? "Your microphone"
let who = app?.name ?? "Your microphone"
warn("◆ \(who) is in use" + (title.map { " · \($0)" } ?? ""))
if let session = self.session {
// A quiet gap followed by the same capture pid can be a device
Expand All @@ -696,14 +695,14 @@ final class Daemon: NSObject, NSApplicationDelegate {
// the user is looking at. Both need it to also end the recording
// this event may have just started.
let ignoreApp: @MainActor () -> Void = { [weak self, weak detector] in
guard let bundleID else { return }
guard let app else { return }
Config.update { config in
var list = config["meeting_excluded_apps"] as? [String] ?? []
if !list.contains(bundleID) { list.append(bundleID) }
if !list.contains(app.bundleID) { list.append(app.bundleID) }
config["meeting_excluded_apps"] = list
}
warn("◇ \(appName ?? bundleID) added to the ignore list")
detector?.declineCurrentMeeting()
warn("◇ \(app.name) added to the ignore list")
detector?.ignoreCurrentClient()
self?.stopSessionIfAutoStarted()
}

Expand All @@ -714,10 +713,10 @@ final class Daemon: NSObject, NSApplicationDelegate {
guard self.session != nil else { return }
detector?.acceptCurrentMeeting()
showToast(
title: appName.map { "Recording \($0) call" } ?? "Recording meeting",
title: app.map { "Recording \($0.name) call" } ?? "Recording meeting",
body: title ?? "Stop from the menu bar or here",
button: "Stop",
secondaryButton: bundleID != nil ? "Ignore" : nil,
secondaryButton: app.map { _ in "Ignore" },
onSecondary: ignoreApp
) { [weak self, weak detector] in
detector?.declineCurrentMeeting()
Expand All @@ -727,12 +726,13 @@ final class Daemon: NSObject, NSApplicationDelegate {
}

askUser(
title: appName.map { "\($0) is in a call" } ?? "Your microphone is in use",
title: app.map { "\($0.name) is in a call" } ?? "Your microphone is in use",
body: title ?? "Record this meeting?",
button: "Record",
// Only when we have both a name to show and an identity to
// store; a process with no app bundle gets Dismiss and no more.
secondaryButton: appName.flatMap { bundleID != nil ? "Ignore \($0)" : nil },
// Only when macOS names the client at all: a bare executable
// with no bundle behind it has no identity we could store, so
// it gets Dismiss and no more.
secondaryButton: app.map { "Ignore \($0.name)" },
onSecondary: ignoreApp,
onDismiss: { detector?.declineCurrentMeeting() }
) { [weak self] in
Expand Down
79 changes: 65 additions & 14 deletions Sources/yap/Detection/MeetingDetector.swift
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ import Foundation
/// device read.
@MainActor
final class MeetingDetector {
/// Something took the mic. The arguments are its capture pid and a display
/// name ("Microsoft Teams") when that pid belongs to an app.
var onMeetingStart: ((pid_t, String?) -> Void)?
/// Something took the mic. The arguments are its capture pid and who it
/// belongs to, when macOS can say — see `MeetingTitle.app`.
var onMeetingStart: ((pid_t, MeetingApp?) -> Void)?

/// The mic went quiet a moment ago (~2 s). The call could still come back
/// from a device switch, so this is only for things that are cheap to
Expand Down Expand Up @@ -71,6 +71,10 @@ final class MeetingDetector {
/// The process the user said no to. A dropout of that same process stays
/// declined, but a different app taking the mic is a different call.
private var declinedPID: pid_t?
/// Processes the caller told us to pretend aren't there — an app the user
/// excluded. Held by pid and dropped when the mic goes quiet, so removing
/// an exclusion takes effect the next time that app takes the mic.
private var ignoredPIDs: Set<pid_t> = []
private var loggedPollFailure = false

init(capturePID: ((pid_t?) -> pid_t?)? = nil) {
Expand Down Expand Up @@ -100,6 +104,7 @@ final class MeetingDetector {
inMeeting = false
askedPID = nil
declinedPID = nil
ignoredPIDs = []
}

/// The user dismissed the prompt for whoever holds the mic right now.
Expand All @@ -108,6 +113,24 @@ final class MeetingDetector {
declinedPID = observedPID
}

/// The client holding the mic belongs to an app the user excluded: drop it
/// out of the scan entirely, like our own pid, and keep looking.
///
/// Not `declineCurrentMeeting`, which only stops the *prompt*. A declined
/// client stays the one we follow, and following it is a cache hit that
/// short-circuits the scan — so a process that holds an input stream for
/// hours (macOS's speech daemon, a self-updating app's helper) would hide
/// every other app behind it, and no real call could be detected for as
/// long as it ran. An excluded app is meant to be invisible to meeting
/// logic, not merely unprompted, and this is what makes that true.
func ignoreCurrentClient() {
guard let pid = observedPID else { return }
ignoredPIDs.insert(pid)
capturing = nil
observedPID = nil
consecutiveActive = 0
}

/// The user accepted the prompt for the client seen on the latest poll.
func acceptCurrentMeeting() {
acceptedPID = observedPID ?? askedPID
Expand Down Expand Up @@ -181,24 +204,30 @@ final class MeetingDetector {
// about, and never for one the user turned down.
guard pid != askedPID, pid != declinedPID else { return }
askedPID = pid
onMeetingStart?(pid, Self.appName(forPID: pid))
onMeetingStart?(pid, MeetingTitle.app(forPID: pid, audioBundleID: audioBundleID(of: pid)))
}

/// The capturing process to follow, excluding yap itself.
///
/// Three tiers, cheapest first, because this runs every second forever:
/// ask the devices whether anyone at all is capturing (~0.1 ms, and the
/// ask the devices whether anyone at all is capturing (0.11 ms, and the
/// answer is no all day), then re-check the client we already know
/// about (~2 ms), and only scan every client when neither settles it.
/// Interrogating all ~50 audio clients costs ~45 ms — that is the main
/// thread, so it stays off the common path.
/// about (0.74 ms), and only scan every client when neither settles it.
/// Interrogating every audio client costs 3.8 ms at 32 of them (M4, p50)
/// and grows with the machine's — that is the main thread, so it stays off
/// the common path.
///
/// Before a prompt is accepted, `preferred` is nil and any external input
/// client can start a meeting. Afterwards it is the pid behind that prompt:
/// clients opened by yap's own recording cannot prolong the call.
private func currentCapturingPID(preferred: pid_t?) -> pid_t? {
guard anyInputDeviceRunning() else {
capturing = nil
// The mic is genuinely idle, so exclusions get re-read from
// config the next time an app takes it. A pid recycled onto a
// different app before that would be ignored in its place; it
// costs one undetected call and heals at the next quiet gap.
ignoredPIDs = []
return nil
}
// Same client as last poll? Confirm the id wasn't recycled onto another
Expand All @@ -225,6 +254,9 @@ final class MeetingDetector {
// the keypress, under its own pid. `suppressed` handles that, and
// the two are easy to confuse when a prompt appears.
guard let pid = pidProperty(object), pid != ownPID else { continue }
// An app the user excluded is invisible here rather than merely
// unprompted, so the scan carries on to whatever else has the mic.
guard !ignoredPIDs.contains(pid) else { continue }
guard preferred == nil || pid == preferred else { continue }
capturing = (object, pid)
return pid
Expand Down Expand Up @@ -264,12 +296,15 @@ final class MeetingDetector {
return AudioObjectGetPropertyDataSize(device, &address, 0, nil, &size) == noErr && size > 0
}

/// Finder's name for the app owning `pid` — "Google Chrome" for a renderer
/// buried in Chrome's Frameworks directory, since the outermost `.app` is
/// the one a human would recognise. Daemons and XPC services have no `.app`
/// and get no name.
private static func appName(forPID pid: pid_t) -> String? {
MeetingTitle.appBundlePath(forPID: pid).map(FileManager.default.displayName(atPath:))
/// Core Audio's own attribution for the client we are following, read only
/// when we are about to prompt: 0.02 ms, and for a helper process or a
/// system daemon it is the only identity there is. See `MeetingTitle.app`.
///
/// Nil when the pid came from the test seam rather than a scan, which
/// leaves identity to the executable path alone.
private func audioBundleID(of pid: pid_t) -> String? {
guard let capturing, capturing.pid == pid else { return nil }
return stringProperty(capturing.object, kAudioProcessPropertyBundleID)
}

// MARK: - Core Audio plumbing
Expand Down Expand Up @@ -335,6 +370,22 @@ final class MeetingDetector {
return value
}

/// A CFString property, transferred out as a Swift string. Core Audio
/// hands back a +1 reference, so it is taken as retained.
private func stringProperty(
_ object: AudioObjectID,
_ selector: AudioObjectPropertySelector
) -> String? {
var address = Self.globalAddress(selector)
var value: Unmanaged<CFString>?
var size = UInt32(MemoryLayout<Unmanaged<CFString>?>.size)
let status = withUnsafeMutablePointer(to: &value) {
AudioObjectGetPropertyData(object, &address, 0, nil, &size, $0)
}
guard status == noErr, let string = value?.takeRetainedValue() else { return nil }
return string as String
}

private static func globalAddress(
_ selector: AudioObjectPropertySelector
) -> AudioObjectPropertyAddress {
Expand Down
76 changes: 63 additions & 13 deletions Sources/yap/Detection/MeetingTitle.swift
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,24 @@ import AppKit
import ApplicationServices
import Darwin

/// Best-effort meeting titles from the window-owning app behind a capture pid.
/// Who is on the microphone: an identity stable enough to keep in the
/// exclusion list, and a name to put in front of the user.
struct MeetingApp {
/// Bundle id of the outermost app bundle behind the capture client, or the
/// identifier Core Audio attributes the stream to when there is no bundle.
let bundleID: String
/// What to call it on screen.
let name: String
}

/// Who is on the microphone and what their meeting is called: an identity for
/// the capture pid, and a best-effort title from the window-owning app behind
/// it.
///
/// Cost contract: call only when prompting, accepting, or resuming after a
/// quiet gap — never from the detector poll loop or dictation path. Each call
/// makes a handful of AX round-trips bounded by a 0.25 second timeout.
/// quiet gap — never from the detector poll loop or dictation path. The title
/// lookups each make a handful of AX round-trips bounded by a 0.25 second
/// timeout.
@MainActor
enum MeetingTitle {
/// Path to the outermost app bundle holding the pid's executable.
Expand All @@ -20,19 +33,56 @@ enum MeetingTitle {
guard let path = String(bytes: buffer[..<Int(length)], encoding: .utf8) else {
return nil
}
let components = (path as NSString).pathComponents
guard let end = components.firstIndex(where: { $0.hasSuffix(".app") }) else { return nil }
return NSString.path(withComponents: Array(components[...end]))
return outermostAppBundle(in: path)
}

/// Bundle identifier of the app behind a capture pid, for the exclusion
/// list. Reading Info.plist off the bundle path — no TCC, no AX, and it
/// works for sandboxed and hardened apps alike.
/// Identify the app behind a capture pid, for the prompt's name and the
/// exclusion list's key. Two routes to the same answer — the outermost
/// `.app` bundle around the client — because either one can come up empty.
///
/// The executable path is free and usually enough, but it fails in both
/// directions: `proc_pidpath` returns nothing at all for a process whose
/// binary was replaced under it (measured on a self-updating app whose
/// helper held the mic), and a daemon like macOS's own `corespeechd` has
/// no `.app` around it to find.
///
/// A process with no `.app` around it has no identity we could store, so
/// it can never be excluded and never grows an Ignore button.
static func bundleID(forPID pid: pid_t) -> String? {
appBundlePath(forPID: pid).flatMap { Bundle(path: $0)?.bundleIdentifier }
/// Core Audio answers where the path doesn't. It attributes every input
/// stream to a bundle id of its own — helpers and system daemons
/// included — and the read costs 0.02 ms. Empty only for a bare
/// executable, which genuinely has no identity worth storing: that is the
/// one case left with no name and no Ignore button.
static func app(forPID pid: pid_t, audioBundleID: String?) -> MeetingApp? {
if let app = identify(bundleAt: appBundlePath(forPID: pid)) { return app }
guard let audioBundleID, !audioBundleID.isEmpty else { return nil }
// Resolved through the installed copy and collapsed onto its container,
// so a helper reads as "Google Chrome" rather than "Google Chrome
// Helper": one Ignore then covers every helper the app starts, under
// the same id Settings' app picker would have stored.
let installed = NSWorkspace.shared.urlForApplication(withBundleIdentifier: audioBundleID)
if let app = identify(bundleAt: installed.flatMap { outermostAppBundle(in: $0.path) }) {
return app
}
// A daemon: nothing installed to resolve and no name to look up, but
// the identifier is stable, so it can still be excluded. Its last
// component is the closest thing to a name it has.
return MeetingApp(
bundleID: audioBundleID,
name: audioBundleID.components(separatedBy: ".").last ?? audioBundleID
)
}

/// Bundle id and Finder name of an app bundle path, if it has both.
private static func identify(bundleAt path: String?) -> MeetingApp? {
guard let path, let bundleID = Bundle(path: path)?.bundleIdentifier else { return nil }
return MeetingApp(bundleID: bundleID, name: FileManager.default.displayName(atPath: path))
}

/// The first `.app` on a path — the one a human would recognise, so a
/// renderer buried in Chrome's Frameworks directory reads as Chrome.
private static func outermostAppBundle(in path: String) -> String? {
let components = (path as NSString).pathComponents
guard let end = components.firstIndex(where: { $0.hasSuffix(".app") }) else { return nil }
return NSString.path(withComponents: Array(components[...end]))
}

/// Best-effort meeting name from the capturing app's windows.
Expand Down
2 changes: 1 addition & 1 deletion Sources/yap/Yap.swift
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ struct Yap: ParsableCommand {
// The single source of truth for what this binary is. Nothing else
// holds a version constant, and the release workflow greps this
// literal against the tag, so a build can never claim the wrong one.
version: "0.3.0",
version: "0.3.1",
// Two subcommands, and that is the product: `run` is the app, `bench`
// is the only thing a terminal can do that the menu bar cannot.
// Everything else — setup, permissions, the login item, the model,
Expand Down
Loading