如何在Python中获得当前的CPU和RAM使用率?


333

在Python中获取当前系统状态(当前CPU,RAM,可用磁盘空间等)的首选方式是什么?* nix和Windows平台的奖励积分。

似乎有几种方法可以从我的搜索中提取出来:

  1. 使用PSI之类的库(目前似乎尚未积极开发并且在多个平台上不受支持)或pystatgrab之类的(自2007年以来一直没有活动,它似乎也不支持Windows)。

  2. 使用平台特定的代码,例如os.popen("ps")在* nix系统和MEMORYSTATUSin中使用a 或类似代码ctypes.windll.kernel32(请参阅ActiveState上的此食谱对于Windows平台使用)。可以将Python类与所有这些代码段放在一起。

并不是说这些方法不好,而是已经有一种受支持的,跨平台的方法来做同样的事情?


您可以使用动态导入来构建自己的多平台库:“如果sys.platform =='win32':将win_sysstatus导入为sysstatus;否则” ...
John Fouhy

1
拥有在App Engine上也可以使用的功能会很酷。
Attila O.

包装的年龄重要吗?如果有人第一次把它们弄对了,为什么他们还不对呢?
保罗·史密斯

Answers:


408

psutil库将为您提供各种平台上的一些系统信息(CPU /内存使用情况):

psutil是一个模块,提供了一个接口,该接口通过使用Python以可移植的方式检索有关正在运行的进程和系统利用率(CPU,内存)的信息,实现了ps,top和Windows任务管理器等工具提供的许多功能。

它当前支持32位和64位体系结构的Linux,Windows,OSX,Sun Solaris,FreeBSD,OpenBSD和NetBSD,Python版本从2.6到3.5(Python 2.4和2.5的用户可以使用2.1.3版本)。


更新:这是一些示例用法psutil

#!/usr/bin/env python
import psutil
# gives a single float value
psutil.cpu_percent()
# gives an object with many fields
psutil.virtual_memory()
# you can convert that object to a dictionary 
dict(psutil.virtual_memory()._asdict())

33
为我工作在OSX:$ pip install psutil; >>> import psutil; psutil.cpu_percent()>>> psutil.virtual_memory()返回一个不错的vmem对象:vmem(total=8589934592L, available=4073336832L, percent=52.6, used=5022085120L, free=3560255488L, active=2817949696L, inactive=513081344L, wired=1691054080L)
滚刀

12
没有psutil库,怎么办?
BigBrownBear00 2015年

2
@ user1054424 python中有一个名为resource的内置库。但是,您似乎最多可以用它来抢夺单个python进程正在使用的内存和/或它的子进程。它似乎也不是很准确。快速测试显示,我的Mac实用工具中的资源减少了大约2MB。
奥斯丁

12
@ BigBrownBear00只是检查psutil的来源;)
Mehulkumar

1
@Jon Cage嗨,Jon,我可以和您一起检查可用内存与可用内存之间的区别吗?我打算使用psutil.virtual_memory()来确定我可以将多少数据加载到内存中进行分析。谢谢你的帮助!
AiRiFiEd

66

使用psutil库。在Ubuntu 18.04上,截至2019年1月30日,pip安装了5.5.0(最新版本)。较旧的版本可能会有所不同。您可以通过在Python中执行以下操作来检查psutil的版本:

from __future__ import print_function  # for Python2
import psutil
print(psutil.__versi‌​on__)

要获取一些内存和CPU统计信息:

from __future__ import print_function
import psutil
print(psutil.cpu_percent())
print(psutil.virtual_memory())  # physical memory usage
print('memory % used:', psutil.virtual_memory()[2])

所述virtual_memory(元组)将具有%的内存使用的全系统。在Ubuntu 18.04上,这似乎被我高估了几个百分点。

您还可以获取当前Python实例使用的内存:

import os
import psutil
pid = os.getpid()
py = psutil.Process(pid)
memoryUse = py.memory_info()[0]/2.**30  # memory use in GB...I think
print('memory use:', memoryUse)

这给出了您的Python脚本的当前内存使用情况。

pypi的pypi页面上有一些更深入的示例。


31

仅适用于Linux:仅使用stdlib依赖项就可以保证RAM使用情况:

import os
tot_m, used_m, free_m = map(int, os.popen('free -t -m').readlines()[-1].split()[1:])

编辑:指定解决方案操作系统依赖性


1
很有用!要以人类可读的单位直接获取它:os.popen('free -th').readlines()[-1].split()[1:]。请注意,此行返回字符串列表。
iipr

python:3.8-slim-buster没有free
马丁托马

21

下面的代码,没有外部库为我工作。我在Python 2.7.9上进行了测试

CPU使用率

import os

    CPU_Pct=str(round(float(os.popen('''grep 'cpu ' /proc/stat | awk '{usage=($2+$4)*100/($2+$4+$5)} END {print usage }' ''').readline()),2))

    #print results
    print("CPU Usage = " + CPU_Pct)

和Ram使用情况,总计,二手和免费

import os
mem=str(os.popen('free -t -m').readlines())
"""
Get a whole line of memory output, it will be something like below
['             total       used       free     shared    buffers     cached\n', 
'Mem:           925        591        334         14         30        355\n', 
'-/+ buffers/cache:        205        719\n', 
'Swap:           99          0         99\n', 
'Total:        1025        591        434\n']
 So, we need total memory, usage and free memory.
 We should find the index of capital T which is unique at this string
"""
T_ind=mem.index('T')
"""
Than, we can recreate the string with this information. After T we have,
"Total:        " which has 14 characters, so we can start from index of T +14
and last 4 characters are also not necessary.
We can create a new sub-string using this information
"""
mem_G=mem[T_ind+14:-4]
"""
The result will be like
1025        603        422
we need to find first index of the first space, and we can start our substring
from from 0 to this index number, this will give us the string of total memory
"""
S1_ind=mem_G.index(' ')
mem_T=mem_G[0:S1_ind]
"""
Similarly we will create a new sub-string, which will start at the second value. 
The resulting string will be like
603        422
Again, we should find the index of first space and than the 
take the Used Memory and Free memory.
"""
mem_G1=mem_G[S1_ind+8:]
S2_ind=mem_G1.index(' ')
mem_U=mem_G1[0:S2_ind]

mem_F=mem_G1[S2_ind+8:]
print 'Summary = ' + mem_G
print 'Total Memory = ' + mem_T +' MB'
print 'Used Memory = ' + mem_U +' MB'
print 'Free Memory = ' + mem_F +' MB'

1
你不觉得grepawk将在Python的字符串处理更好的照顾?
Reinderien '18 -10-22

我个人不熟悉awk,因此在下面制作了cp用法摘要的纯真版本。非常方便,谢谢!
周杰伦

3
可以断言此代码不使用外部库。实际上,它们对grep,awk和free的可用性有严格的依赖性。这使得上面的代码不可移植。OP指出“ * nix和Windows平台的奖励积分”。
莱普顿船长

10

这是我前几天整理的东西,仅是Windows,但可以帮助您获得部分所需的工作。

派生自:“用于sys的可用mem” http://msdn2.microsoft.com/zh-cn/library/aa455130.aspx

“单个过程信息和python脚本示例” http://www.microsoft.com/technet/scriptcenter/scripts/default.mspx?mfr=true

注意:WMI界面/过程也可用于执行类似的任务,因为当前的方法可以满足我的需求,所以我在这里不使用它,但是如果有朝一日需要扩展或改进它,则可能需要研究可用的WMI工具。 。

适用于python的WMI:

http://tgolden.sc.sabren.com/python/wmi.html

编码:

'''
Monitor window processes

derived from:
>for sys available mem
http://msdn2.microsoft.com/en-us/library/aa455130.aspx

> individual process information and python script examples
http://www.microsoft.com/technet/scriptcenter/scripts/default.mspx?mfr=true

NOTE: the WMI interface/process is also available for performing similar tasks
        I'm not using it here because the current method covers my needs, but if someday it's needed
        to extend or improve this module, then may want to investigate the WMI tools available.
        WMI for python:
        http://tgolden.sc.sabren.com/python/wmi.html
'''

__revision__ = 3

import win32com.client
from ctypes import *
from ctypes.wintypes import *
import pythoncom
import pywintypes
import datetime


class MEMORYSTATUS(Structure):
    _fields_ = [
                ('dwLength', DWORD),
                ('dwMemoryLoad', DWORD),
                ('dwTotalPhys', DWORD),
                ('dwAvailPhys', DWORD),
                ('dwTotalPageFile', DWORD),
                ('dwAvailPageFile', DWORD),
                ('dwTotalVirtual', DWORD),
                ('dwAvailVirtual', DWORD),
                ]


def winmem():
    x = MEMORYSTATUS() # create the structure
    windll.kernel32.GlobalMemoryStatus(byref(x)) # from cytypes.wintypes
    return x    


class process_stats:
    '''process_stats is able to provide counters of (all?) the items available in perfmon.
    Refer to the self.supported_types keys for the currently supported 'Performance Objects'

    To add logging support for other data you can derive the necessary data from perfmon:
    ---------
    perfmon can be run from windows 'run' menu by entering 'perfmon' and enter.
    Clicking on the '+' will open the 'add counters' menu,
    From the 'Add Counters' dialog, the 'Performance object' is the self.support_types key.
    --> Where spaces are removed and symbols are entered as text (Ex. # == Number, % == Percent)
    For the items you wish to log add the proper attribute name in the list in the self.supported_types dictionary,
    keyed by the 'Performance Object' name as mentioned above.
    ---------

    NOTE: The 'NETFramework_NETCLRMemory' key does not seem to log dotnet 2.0 properly.

    Initially the python implementation was derived from:
    http://www.microsoft.com/technet/scriptcenter/scripts/default.mspx?mfr=true
    '''
    def __init__(self,process_name_list=[],perf_object_list=[],filter_list=[]):
        '''process_names_list == the list of all processes to log (if empty log all)
        perf_object_list == list of process counters to log
        filter_list == list of text to filter
        print_results == boolean, output to stdout
        '''
        pythoncom.CoInitialize() # Needed when run by the same process in a thread

        self.process_name_list = process_name_list
        self.perf_object_list = perf_object_list
        self.filter_list = filter_list

        self.win32_perf_base = 'Win32_PerfFormattedData_'

        # Define new datatypes here!
        self.supported_types = {
                                    'NETFramework_NETCLRMemory':    [
                                                                        'Name',
                                                                        'NumberTotalCommittedBytes',
                                                                        'NumberTotalReservedBytes',
                                                                        'NumberInducedGC',    
                                                                        'NumberGen0Collections',
                                                                        'NumberGen1Collections',
                                                                        'NumberGen2Collections',
                                                                        'PromotedMemoryFromGen0',
                                                                        'PromotedMemoryFromGen1',
                                                                        'PercentTimeInGC',
                                                                        'LargeObjectHeapSize'
                                                                     ],

                                    'PerfProc_Process':              [
                                                                          'Name',
                                                                          'PrivateBytes',
                                                                          'ElapsedTime',
                                                                          'IDProcess',# pid
                                                                          'Caption',
                                                                          'CreatingProcessID',
                                                                          'Description',
                                                                          'IODataBytesPersec',
                                                                          'IODataOperationsPersec',
                                                                          'IOOtherBytesPersec',
                                                                          'IOOtherOperationsPersec',
                                                                          'IOReadBytesPersec',
                                                                          'IOReadOperationsPersec',
                                                                          'IOWriteBytesPersec',
                                                                          'IOWriteOperationsPersec'     
                                                                      ]
                                }

    def get_pid_stats(self, pid):
        this_proc_dict = {}

        pythoncom.CoInitialize() # Needed when run by the same process in a thread
        if not self.perf_object_list:
            perf_object_list = self.supported_types.keys()

        for counter_type in perf_object_list:
            strComputer = "."
            objWMIService = win32com.client.Dispatch("WbemScripting.SWbemLocator")
            objSWbemServices = objWMIService.ConnectServer(strComputer,"root\cimv2")

            query_str = '''Select * from %s%s''' % (self.win32_perf_base,counter_type)
            colItems = objSWbemServices.ExecQuery(query_str) # "Select * from Win32_PerfFormattedData_PerfProc_Process")# changed from Win32_Thread        

            if len(colItems) > 0:        
                for objItem in colItems:
                    if hasattr(objItem, 'IDProcess') and pid == objItem.IDProcess:

                            for attribute in self.supported_types[counter_type]:
                                eval_str = 'objItem.%s' % (attribute)
                                this_proc_dict[attribute] = eval(eval_str)

                            this_proc_dict['TimeStamp'] = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.') + str(datetime.datetime.now().microsecond)[:3]
                            break

        return this_proc_dict      


    def get_stats(self):
        '''
        Show process stats for all processes in given list, if none given return all processes   
        If filter list is defined return only the items that match or contained in the list
        Returns a list of result dictionaries
        '''    
        pythoncom.CoInitialize() # Needed when run by the same process in a thread
        proc_results_list = []
        if not self.perf_object_list:
            perf_object_list = self.supported_types.keys()

        for counter_type in perf_object_list:
            strComputer = "."
            objWMIService = win32com.client.Dispatch("WbemScripting.SWbemLocator")
            objSWbemServices = objWMIService.ConnectServer(strComputer,"root\cimv2")

            query_str = '''Select * from %s%s''' % (self.win32_perf_base,counter_type)
            colItems = objSWbemServices.ExecQuery(query_str) # "Select * from Win32_PerfFormattedData_PerfProc_Process")# changed from Win32_Thread

            try:  
                if len(colItems) > 0:
                    for objItem in colItems:
                        found_flag = False
                        this_proc_dict = {}

                        if not self.process_name_list:
                            found_flag = True
                        else:
                            # Check if process name is in the process name list, allow print if it is
                            for proc_name in self.process_name_list:
                                obj_name = objItem.Name
                                if proc_name.lower() in obj_name.lower(): # will log if contains name
                                    found_flag = True
                                    break

                        if found_flag:
                            for attribute in self.supported_types[counter_type]:
                                eval_str = 'objItem.%s' % (attribute)
                                this_proc_dict[attribute] = eval(eval_str)

                            this_proc_dict['TimeStamp'] = datetime.datetime.now().strftime('%Y-%m-%d %H:%M:%S.') + str(datetime.datetime.now().microsecond)[:3]
                            proc_results_list.append(this_proc_dict)

            except pywintypes.com_error, err_msg:
                # Ignore and continue (proc_mem_logger calls this function once per second)
                continue
        return proc_results_list     


def get_sys_stats():
    ''' Returns a dictionary of the system stats'''
    pythoncom.CoInitialize() # Needed when run by the same process in a thread
    x = winmem()

    sys_dict = { 
                    'dwAvailPhys': x.dwAvailPhys,
                    'dwAvailVirtual':x.dwAvailVirtual
                }
    return sys_dict


if __name__ == '__main__':
    # This area used for testing only
    sys_dict = get_sys_stats()

    stats_processor = process_stats(process_name_list=['process2watch'],perf_object_list=[],filter_list=[])
    proc_results = stats_processor.get_stats()

    for result_dict in proc_results:
        print result_dict

    import os
    this_pid = os.getpid()
    this_proc_results = stats_processor.get_pid_stats(this_pid)

    print 'this proc results:'
    print this_proc_results

http://monkut.webfactional.com/blog/archive/2009/1/21/windows-process-memory-logging-python


使用GlobalMemoryStatusEx而不是GlobalMemoryStatus,因为旧的可能返回错误的值。
phobie,2012年

7
您应该避免from x import *声明!它们使主命名空间混乱,并覆盖其他函数和变量。
phobie,2012年

6

我们之所以选择使用常规信息源,是因为我们可以发现空闲内存中的瞬时波动,并且认为查询meminfo数据源很有帮助。这也帮助我们获得了一些预先准备的相关参数。

import os

linux_filepath = "/proc/meminfo"
meminfo = dict(
    (i.split()[0].rstrip(":"), int(i.split()[1]))
    for i in open(linux_filepath).readlines()
)
meminfo["memory_total_gb"] = meminfo["MemTotal"] / (2 ** 20)
meminfo["memory_free_gb"] = meminfo["MemFree"] / (2 ** 20)
meminfo["memory_available_gb"] = meminfo["MemAvailable"] / (2 ** 20)

输出供参考(我们删除了所有换行符以进行进一步分析)

内存总数:1014500 kB内存空闲:562680 kB可用内存:646364 kB缓冲区:15144 kB缓存:210720 kB交换缓存:0 kB活动:261476 kB非活动:128888 kB活动(匿名):167092 kB非活动(匿名):20888 kB活动(文件) :94384 kB无效(文件):108000 kB无法启动:3652 kB锁定:3652 kB交换总量:0 kB交换免费:0 kB脏污:0 kB写回:0 kB AnonPages:168160 kB Mapped:81352 kB Shmem:21060 kB Slab:34492 kB SReclaimable:18044 kB SUnreclaim:16448 kB KernelStack:2672 kB PageTables:8180 kB NFS_Unstable:0 kB Bounce:0 kB WritebackTmp:0 kB CommitLimit:507248 kB Committed_AS:1038756 kB VmallocTotal:34359738367 kB kB Vmallocd: 0 kB AnonHugePages:88064 kB Cma总计:0 kB CmaFree:0 kB HugePages_Total:0 HugePages_Free:0 HugePages_Rsvd:0 HugePages_Surp:0 HugePagesize:2048 kB DirectMap4k:43008 kB DirectMap2M:1005568 kB


似乎无法按预期工作:stackoverflow.com/q/61498709/562769
Martin Thoma

4

我觉得这些答案是针对Python 2编写的,无论如何都没有人提及resource可用于Python 3 的标准软件包。它提供了用于获取给定进程(默认情况下为调用Python进程)的资源限制的命令。这与整个系统当前对资源的使用情况不同,但是可以解决一些相同的问题,例如“我想确保此脚本只使用X个RAM。”


3

“ ...当前系统状态(当前CPU,RAM,可用磁盘空间等)”和“ * nix和Windows平台”可能很难实现。

操作系统在管理这些资源的方式上根本不同。实际上,它们在核心概念方面有所不同,例如定义什么才算是系统和什么才算是应用程序时间。

“可用磁盘空间”?什么算作“磁盘空间”?所有设备的所有分区?多重引导环境中的外部分区呢?

我认为Windows和* nix之间没有足够清晰的共识使之成为可能。实际上,在称为Windows的各种操作系统之间甚至可能没有达成共识。是否有一个适用于XP和Vista的Windows API?


4
df -h在Windows和* nix上都回答“磁盘空间”问题。
jfs

4
@JFSebastian:哪个Windows?我收到“ df”无法识别... Windows XP Pro的错误消息。我想念什么?
S.Lott

3
您也可以在Windows上安装新程序。
jfs 2015年

2

此脚本用于CPU使用率:

import os

def get_cpu_load():
    """ Returns a list CPU Loads"""
    result = []
    cmd = "WMIC CPU GET LoadPercentage "
    response = os.popen(cmd + ' 2>&1','r').read().strip().split("\r\n")
    for load in response[1:]:
       result.append(int(load))
    return result

if __name__ == '__main__':
    print get_cpu_load()

1
  • 有关CPU的详细信息,请使用psutil

    https://psutil.readthedocs.io/en/latest/#cpu

  • 对于RAM频率(以MHz为单位),请使用内置的Linux库dmidecode并稍微控制输出;)。此命令需要root权限,因此也要提供密码。只需复制以下命令,将mypass替换为您的密码

import os

os.system("echo mypass | sudo -S dmidecode -t memory | grep 'Clock Speed' | cut -d ':' -f2")

-------------------输出---------------------------
1600 MT / s
未知
1600 MT / s
未知0

  • 更具体地说
    [i for i in os.popen("echo mypass | sudo -S dmidecode -t memory | grep 'Clock Speed' | cut -d ':' -f2").read().split(' ') if i.isdigit()]

--------------------------输出----------------------- -
['1600','1600']


添加一些说明
帕拉斯呵叻

1

为了获得程序的逐行存储和时间分析,建议使用memory_profilerline_profiler

安装:

# Time profiler
$ pip install line_profiler
# Memory profiler
$ pip install memory_profiler
# Install the dependency for a faster analysis
$ pip install psutil

共同的部分是,您可以使用相应的修饰符指定要分析的功能。

示例:我的Python文件main.py中有几个要分析的功能。其中之一是linearRegressionfit()。我需要使用装饰器@profile,该装饰器可帮助我针对以下两个方面分析代码:时间和内存。

对函数定义进行以下更改

@profile
def linearRegressionfit(Xt,Yt,Xts,Yts):
    lr=LinearRegression()
    model=lr.fit(Xt,Yt)
    predict=lr.predict(Xts)
    # More Code

对于时间分析

跑:

$ kernprof -l -v main.py

输出量

Total time: 0.181071 s
File: main.py
Function: linearRegressionfit at line 35

Line #      Hits         Time  Per Hit   % Time  Line Contents
==============================================================
    35                                           @profile
    36                                           def linearRegressionfit(Xt,Yt,Xts,Yts):
    37         1         52.0     52.0      0.1      lr=LinearRegression()
    38         1      28942.0  28942.0     75.2      model=lr.fit(Xt,Yt)
    39         1       1347.0   1347.0      3.5      predict=lr.predict(Xts)
    40                                           
    41         1       4924.0   4924.0     12.8      print("train Accuracy",lr.score(Xt,Yt))
    42         1       3242.0   3242.0      8.4      print("test Accuracy",lr.score(Xts,Yts))

对于内存分析

跑:

$ python -m memory_profiler main.py

输出量

Filename: main.py

Line #    Mem usage    Increment   Line Contents
================================================
    35  125.992 MiB  125.992 MiB   @profile
    36                             def linearRegressionfit(Xt,Yt,Xts,Yts):
    37  125.992 MiB    0.000 MiB       lr=LinearRegression()
    38  130.547 MiB    4.555 MiB       model=lr.fit(Xt,Yt)
    39  130.547 MiB    0.000 MiB       predict=lr.predict(Xts)
    40                             
    41  130.547 MiB    0.000 MiB       print("train Accuracy",lr.score(Xt,Yt))
    42  130.547 MiB    0.000 MiB       print("test Accuracy",lr.score(Xts,Yts))

同样,也可以matplotlib使用

$ mprof run main.py
$ mprof plot

在此处输入图片说明 注意:经过测试

line_profiler 版本== 3.0.2

memory_profiler 版本== 0.57.0

psutil 版本== 5.7.0



0

基于@Hrabal的cpu使用代码,这是我使用的:

from subprocess import Popen, PIPE

def get_cpu_usage():
    ''' Get CPU usage on Linux by reading /proc/stat '''

    sub = Popen(('grep', 'cpu', '/proc/stat'), stdout=PIPE, stderr=PIPE)
    top_vals = [int(val) for val in sub.communicate()[0].split('\n')[0].split[1:5]]

    return (top_vals[0] + top_vals[2]) * 100. /(top_vals[0] + top_vals[2] + top_vals[3])

-12

我认为没有可用的受支持的多平台库。请记住,Python本身是用C编写的,因此,任何库都将像上面建议的那样,明智地决定要运行哪个特定于操作系统的代码段。


1
为什么这个答案不好?这句话是假的吗?
EAzevedo

4
因为psutil是一个受良好支持的多平台库,可能满足操作需求
amadain
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.