Blog posts for tags/windows

  1. System load average on Windows in Python

    psutil 5.6.2 is out. It implements an emulation of os.getloadavg() on Windows, kindly contributed by Ammar Askar, who originally implemented it for CPython's test suite.

    This idea has been floating around for quite a while. The first proposal dates back to 2010, when psutil was still hosted on Google Code, and it popped up multiple times over the years. There's a bunch of info online mentioning the pieces you'd theoretically use (the so-called System Processor Queue Length), but I couldn't find any real implementation. A quick search suggests there's real demand for this, but very few tools provide it natively (the only ones I could find are sFlowTrend and Zabbix). So I'm glad this finally landed in psutil / Python.

    Other improvements and bugfixes in psutil 5.6.2

    The full list is in the changelog. A couple worth mentioning:

    • #1476: ability to set a process's high I/O priority on Windows.
    • #1458: colorized test output. Nobody will use this directly, but it's nice and I'm porting it to other projects I maintain (e.g. pyftpdlib). Good candidate for a small PyPI module that could also include the unittest extensions I've been re-implementing piece by piece:
      • #1478: re-running failed tests.
      • display test timings / durations. This is something I'm also contributing to CPython: BPO-4080 and PR-12271.

    About me

    I'm currently in China (Shenzhen) for a mix of vacation and work, and I will likely take a break from Open Source for a while (about 2.5 months), during which I'll also go to the Philippines and Japan.

    External

  2. Announcing psutil 5.6.0

    psutil 5.6.0 is out. Highlights: a new Process.parents() method, several important Windows improvements, and the removal of Process.memory_maps() on macOS.

    Process parents()

    The new method returns the parents of a process as a list of Process instances. If no parents are known, an empty list is returned.

    >>> import psutil
    >>> p = psutil.Process(5312)
    >>> p.parents()
    [psutil.Process(pid=4699, name='bash', started='09:06:44'),
     psutil.Process(pid=4689, name='gnome-terminal-server', started='09:06:44'),
     psutil.Process(pid=1, name='systemd', started='05:56:55')]
    

    Nothing fundamentally new here, since this is a convenience wrapper around Process.parent(), but it's still nice to have it built in. It pairs well with Process.children() when working with process trees. The idea was proposed by Ghislain Le Meur.

    Windows

    Certain Windows APIs that need to be dynamically loaded from DLLs are now loaded only once at startup, instead of on every function call. This makes some operations 50% to 100% faster; see benchmarks in PR-1422.

    Process.suspend() and Process.resume() previously iterated over all process threads via CreateToolhelp32Snapshot(), which was unorthodox and broke when the process had been suspended by Process Hacker. They now call the undocumented NtSuspendProcess() / NtResumeProcess() NT APIs, same as Process Hacker and Sysinternals tools. Discussed in #1379, implemented in PR-1435.

    SE DEBUG is a privilege bit set on the Python process at startup so psutil can query processes owned by other users (Administrator, Local System), meaning fewer AccessDenied exceptions for low-PID processes. The code setting it had presumably been broken for years and is now finally fixed in PR-1429.

    Removal of Process.memory_maps() on macOS

    Process.memory_maps() is gone on macOS (#1291). The underlying Apple API would randomly raise EINVAL or segfault the host process, and no amount of reverse-engineering produced a safe fix. So I removed it. This is covered in a separate post.

    Improved exceptions

    One problem that affected psutil maintenance over the years was receiving bug reports whose tracebacks did not indicate which syscall had actually failed. This was especially painful on Windows, where a single routine may invoke multiple Windows APIs. Now the OSError (or WindowsError) exception includes the syscall from which the error originated. See PR-1428.

    Other changes

    See the changelog.

  3. Windows services support

    New psutil 4.2.0 is out. The highlight of this release is the support for Windows services (executables that run at system startup, similar to UNIX init scripts):

    >>> import psutil
    >>> list(psutil.win_service_iter())
    [<WindowsService(name='AeLookupSvc', display_name='Application Experience') at 38850096>,
     <WindowsService(name='ALG', display_name='Application Layer Gateway Service') at 38850128>,
     <WindowsService(name='APNMCP', display_name='Ask Update Service') at 38850160>,
     <WindowsService(name='AppIDSvc', display_name='Application Identity') at 38850192>,
     ...]
    >>> s = psutil.win_service_get('alg')
    >>> s.as_dict()
    {'binpath': 'C:\\Windows\\System32\\alg.exe',
     'description': 'Provides support for 3rd party protocol plug-ins for Internet Connection Sharing',
     'display_name': 'Application Layer Gateway Service',
     'name': 'alg',
     'pid': None,
     'start_type': 'manual',
     'status': 'stopped',
     'username': 'NT AUTHORITY\\LocalService'}
    

    I decided to do this mainly because I find pywin32 APIs too low levelish. Having something like this in psutil can be useful to discover and monitor services more easily. The code was implemented in PR-803. The API for querying a service is similar to psutil.Process. You can get a reference to a service object by using its name (which is unique for every service) and then use methods like WindowsService.name() and WindowsService.status():

    >>> s = psutil.win_service_get('alg')
    >>> s.name()
    'alg'
    >>> s.status()
    'stopped'
    

    Initially I thought about providing a full set of APIs to handle all aspects of service management, including start(), stop(), restart(), install(), uninstall() and modify(). However, I soon realized I would have ended up reimplementing what pywin32 already provides, at the cost of overcrowding the psutil API (see my reasoning here). I think psutil really focuses on monitoring, not on installing and modifying system components, especially something as critical as a Windows service.

    Considerations about Windows services

    Typically, a Windows service is an executable (.exe) that runs at system startup and continues running in the background. It is roughly the equivalent of a UNIX init script. All services are controlled by a "manager", which keeps track of their status and metadata (e.g. description, startup type). It is interesting to note that since (most) services are bound to an executable (and hence a process) you can reference them via their process PID:

    >>> s = psutil.win_service_get('sshd')
    >>> s
    <WindowsService(name='sshd', display_name='Open SSH server') at 38853046>
    >>> s.pid()
    1865
    >>> p = psutil.Process(1865)
    >>> p
    <psutil.Process(pid=19547, name='sshd.exe') at 140461487781328>
    >>> p.exe()
    'C:\CygWin\bin\sshd'
    

    Other improvements

    psutil 4.2.0 comes with 2 other enhancements for Linux:

    • psutil.virtual_memory() returns a new shared memory field. This is the same value reported by free cmdline utility.
    • I changed how /proc was parsed. Instead of reading /proc/{pid}/status line by line I used a regular expression. Here's the speedups:
      • Process.ppid() ~20% faster.
      • Process.status() ~28% faster.
      • Process.name() ~25% faster.
      • Process.num_threads() ~20% faster (on Python 3 only; on Python 2 it's a bit slower; I suppose the re module received some improvements).

    Discussion

Social

Feed