Blog posts for tags/macos

  1. Announcing psutil 5.6.0

    psutil 5.6.0 is out. Highlights: a new Process.parents() method, several important Windows improvements, and the removal of Process.memory_maps() on macOS.

    Process parents()

    The new method returns the parents of a process as a list of Process instances. If no parents are known, an empty list is returned.

    >>> import psutil
>>> p = psutil.Process(5312)
>>> p.parents()
[psutil.Process(pid=4699, name='bash', started='09:06:44'),
 psutil.Process(pid=4689, name='gnome-terminal-server', started='09:06:44'),
 psutil.Process(pid=1, name='systemd', started='05:56:55')]

    Nothing fundamentally new here, since this is a convenience wrapper around Process.parent(), but it's still nice to have it built in. It pairs well with Process.children() when working with process trees. The idea was proposed by Ghislain Le Meur.

    Windows

    Certain Windows APIs that need to be dynamically loaded from DLLs are now loaded only once at startup, instead of on every function call. This makes some operations 50% to 100% faster; see benchmarks in PR-1422.

    Process.suspend() and Process.resume() previously iterated over all process threads via CreateToolhelp32Snapshot(), which was unorthodox and broke when the process had been suspended by Process Hacker. They now call the undocumented NtSuspendProcess() / NtResumeProcess() NT APIs, same as Process Hacker and Sysinternals tools. Discussed in #1379, implemented in PR-1435.

    SE DEBUG is a privilege bit set on the Python process at startup so psutil can query processes owned by other users (Administrator, Local System), meaning fewer AccessDenied exceptions for low-PID processes. The code setting it had presumably been broken for years and is now finally fixed in PR-1429.

    Removal of Process.memory_maps() on macOS

    Process.memory_maps() is gone on macOS (#1291). The underlying Apple API would randomly raise EINVAL or segfault the host process, and no amount of reverse-engineering produced a safe fix. So I removed it. This is covered in a separate post.

    Improved exceptions

    One problem that affected psutil maintenance over the years was receiving bug reports whose tracebacks did not indicate which syscall had actually failed. This was especially painful on Windows, where a single routine may invoke multiple Windows APIs. Now the OSError (or WindowsError) exception includes the syscall from which the error originated. See PR-1428.

    Other changes

    See the changelog.

  2. Removing Process.memory_maps() on macOS

    This is part of the psutil 5.6.0 release (see the full release notes).

    As of 5.6.0, Process.memory_maps() is no longer defined on macOS.

    The bug

    #1291: on macOS, Process.memory_maps() would either raise OSError: [Errno 22] Invalid argument or segfault the whole Python process! Both triggered from code as simple as psutil.Process().as_dict(), since Process.as_dict() iterates every attribute, and Process.memory_maps() is one of them.

    The root cause was inside Apple's undocumented proc_regionfilename() syscall. On some memory regions it returns EINVAL. On others it takes the process down. Which regions? Nobody figured out. Arnon Yaari (@wiggin15) did most of the investigation: he wrote a standalone C reproducer and walked me through what he'd tried.

    In PR-1436 I attempted a fix by reverse-engineering vmmap(1) but it didn't work. The fundamental problem is that vmmap is closed source and proc_regionfilename is undocumented. Neither my virtualized macOS (10.11.6) nor Travis CI (10.12.1) could reproduce the bug, which reproduced reliably only on 10.14.3.

    Why remove outright

    While removing the C code I noticed that the macOS unit test had been disabled long ago, presumably by me after recurring flaky Travis runs. Meaning that the method had been broken on some macOS versions far longer than the 2018 bug report suggested.

    Deprecating for a cycle didn't help either: raising AccessDenied breaks code that relied on a successful return, returning an empty list does the same silently, and leaving the method in place doesn't stop the segfault. Basically there was no sane solution, so since 5.6 is a major version I decided to just remove Process.memory_maps() for good.

    On macOS it never supported other processes anyway. Calling it on any PID other than the current one (or its children) raised AccessDenied, even as root.

    If someone finds a Mach API path that works, the method can return. Nobody has found one so far.

Social

Feed