Hacked By AnonymousFox
B
YI~d�h � @ s0 d dgZ ddlZddlZddlZddlZddlZddlZddlZddlm Z ddlm
Z
m Z dZ dZ
dZe�� Zdd � Zd
d � ZG d d
� d
e�ZG dd� d�Zdd� ZG dd� de�Zd$dd�Zdd� ZG dd � d e�ZG dd � d e�Z e Z G d d � d e �Z G d d � d e�Z G d!d"� d"e �Z G d#d� de�Z!dS )%�Pool�
ThreadPool� N� )�util)� get_context� TimeoutError� c C s t t| � �S )N)�list�map)�args� r �9/opt/alt/python37/lib64/python3.7/multiprocessing/pool.py�mapstar+ s r c C s t t�| d | d ��S )Nr r )r � itertools�starmap)r r r r
� starmapstar. s r c @ s e Zd Zdd� Zdd� ZdS )�RemoteTracebackc C s
|| _ d S )N)�tb)�selfr r r r
�__init__6 s zRemoteTraceback.__init__c C s | j S )N)r )r r r r
�__str__8 s zRemoteTraceback.__str__N)�__name__�
__module__� __qualname__r r r r r r
r 5 s r c @ s e Zd Zdd� Zdd� ZdS )�ExceptionWithTracebackc C s0 t �t|�||�}d�|�}|| _d| | _d S )N� z
"""
%s""")� traceback�format_exception�type�join�excr )r r r r r r
r < s
z ExceptionWithTraceback.__init__c C s t | j| jffS )N)� rebuild_excr r )r r r r
�
__reduce__A s z!ExceptionWithTraceback.__reduce__N)r r r r r" r r r r
r ; s r c C s t |�| _| S )N)r � __cause__)r r r r r
r! D s
r! c s, e Zd Z� fdd�Zdd� Zdd� Z� ZS )�MaybeEncodingErrorc s. t |�| _t |�| _tt| ��| j| j� d S )N)�reprr �value�superr$ r )r r r& )� __class__r r
r P s
zMaybeEncodingError.__init__c C s d| j | jf S )Nz(Error sending result: '%s'. Reason: '%s')r& r )r r r r
r U s zMaybeEncodingError.__str__c C s d| j j| f S )Nz<%s: %s>)r( r )r r r r
�__repr__Y s zMaybeEncodingError.__repr__)r r r r r r) �
__classcell__r r )r( r
r$ L s r$ r Fc
C s� |d k r(t |t�r|dks(td�|���|j}| j}t| d�rR| j�� |j �� |d k rb||� d}�xP|d ks�|�r�||k �r�y
|� } W n$ t
t fk
r� t �
d� P Y nX | d kr�t �
d� P | \}
} } }
}yd| |
|�f}W nH tk
�r. } z(|�r| tk �rt||j�}d|f}W d d }~X Y nX y||
| |f� W nR tk
�r� } z2t||d �}t �
d | � ||
| d|ff� W d d }~X Y nX d } }
} } }
}|d7 }qjW t �
d
| � d S ) Nr zMaxtasks {!r} is not valid�_writerr z)worker got EOFError or OSError -- exitingz worker got sentinel -- exitingTFz0Possible encoding error while sending result: %sz worker exiting after %d tasks)�
isinstance�int�AssertionError�format�put�get�hasattrr+ �close�_reader�EOFError�OSErrorr �debug� Exception�_helper_reraises_exceptionr �
__traceback__r$ )�inqueue�outqueue� initializer�initargsZmaxtasksZwrap_exceptionr0 r1 Z completed�task�job�i�funcr �kwds�result�e�wrappedr r r
�worker] sJ
$ rG c C s | �d S )Nr )Zexr r r
r9 � s r9 c @ s( e Zd ZdZdd� Zd=dd�Zdd � Zd
d � Zd d
� Zdd� Z di fdd�Z
d>dd�Z d?dd�Z d@dd�Z
dd� ZdAdd �ZdBd d �Zdi ddfd d �ZdCd!d"�ZdDd#d$�Zed%d&� �Zed'd(� �Zed)d*� �Zed+d,� �Zd-d.� Zd/d0� Zd1d2� Zd3d4� Z ed5d6� �Z e d7d8� �Z d9d:� Z d;d<� Z!dS )Er Tc O s | j j||�S )N)�_ctx�Process)r r rC r r r
rI � s z Pool.ProcessNr c C sl |pt � | _| �� t�� | _i | _t| _|| _ || _
|| _ |d krPt �
� pNd}|dk r`td��|d k rxt|�sxtd��|| _g | _| �� tjtj| fd�| _d| j_t| j_| j�� tjtj| j| j | j | j| jfd�| _ d| j _t| j _| j �� tjtj | j | j | jfd�| _!d| j!_t| j!_| j!�� t"j#| | j$| j| j%| j | j| j| j | j!| jfdd�| _&d S )Nr z&Number of processes must be at least 1z initializer must be a callable)�targetr T� )r Z exitpriority)'r rH �
_setup_queues�queue� SimpleQueue�
_taskqueue�_cache�RUN�_state�_maxtasksperchild� _initializer� _initargs�os� cpu_count�
ValueError�callable� TypeError�
_processes�_pool�_repopulate_pool� threadingZThreadr �_handle_workers�_worker_handler�daemon�start�
_handle_tasks�
_quick_put� _outqueue�
_task_handler�_handle_results�
_quick_get�_result_handlerr ZFinalize�_terminate_pool�_inqueue�
_terminate)r � processesr= r> Zmaxtasksperchild�contextr r r
r � sT
z
Pool.__init__c C sZ d}xPt tt| j���D ]:}| j| }|jd k rt�d| � |�� d}| j|= qW |S )NFzcleaning up worker %dT)�reversed�range�lenr\ �exitcoder r7 r )r ZcleanedrA rG r r r
�_join_exited_workers� s
zPool._join_exited_workersc C s| xvt | jt| j� �D ]^}| jt| j| j| j| j | j
| j fd�}| j� |� |j
�dd�|_
d|_|�� t�d� qW d S )N)rJ r rI Z
PoolWorkerTz added worker)rp r[ rq r\ rI rG rk re rT rU rS �_wrap_exception�append�name�replacera rb r r7 )r rA �wr r r
r] � s zPool._repopulate_poolc C s | � � r| �� d S )N)rs r] )r r r r
�_maintain_pool� s zPool._maintain_poolc C s4 | j �� | _| j �� | _| jjj| _| jjj| _ d S )N)
rH rN rk re r+ �sendrd r4 �recvrh )r r r r
rL � s zPool._setup_queuesc C s | � |||��� S )N)� apply_asyncr1 )r rB r rC r r r
�apply s z
Pool.applyc C s | � ||t|��� S )N)�
_map_asyncr r1 )r rB �iterable� chunksizer r r
r
s zPool.mapc C s | � ||t|��� S )N)r~ r r1 )r rB r r� r r r
r s z Pool.starmapc C s | � ||t|||�S )N)r~ r )r rB r r� �callback�error_callbackr r r
�
starmap_async s zPool.starmap_asyncc
c sn y0d}x&t |�D ]\}}||||fi fV qW W n8 tk
rh } z||d t|fi fV W d d }~X Y nX d S )N���r )� enumerater8 r9 )r Z
result_jobrB r rA �xrE r r r
�_guarded_task_generation s z Pool._guarded_task_generationr c C s� | j tkrtd��|dkrFt| j�}| j�| �|j||�|j f� |S |dk r\td�
|���t � |||�}t| j�}| j�| �|jt
|�|j f� dd� |D �S d S )NzPool not runningr z Chunksize must be 1+, not {0:n}c s s | ]}|D ]
}|V q
qd S )Nr )�.0�chunk�itemr r r
� <genexpr>E s z Pool.imap.<locals>.<genexpr>)rR rQ rX � IMapIteratorrP rO r0 r� �_job� _set_lengthr/ r �
_get_tasksr )r rB r r� rD � task_batchesr r r
�imap) s(
z Pool.imapc C s� | j tkrtd��|dkrFt| j�}| j�| �|j||�|j f� |S |dk r\td�
|���t � |||�}t| j�}| j�| �|jt
|�|j f� dd� |D �S d S )NzPool not runningr z Chunksize must be 1+, not {0!r}c s s | ]}|D ]
}|V q
qd S )Nr )r� r� r� r r r
r� b s z&Pool.imap_unordered.<locals>.<genexpr>)rR rQ rX �IMapUnorderedIteratorrP rO r0 r� r� r� r/ r r� r )r rB r r� rD r� r r r
�imap_unorderedG s&
zPool.imap_unorderedc C sB | j tkrtd��t| j||�}| j�|jd|||fgd f� |S )NzPool not runningr )rR rQ rX � ApplyResultrP rO r0 r� )r rB r rC r� r� rD r r r
r| d s
zPool.apply_asyncc C s | � ||t|||�S )N)r~ r )r rB r r� r� r� r r r
� map_asynco s zPool.map_asyncc
C s� | j tkrtd��t|d�s$t|�}|d krTtt|�t| j�d �\}}|rT|d7 }t|�dkrdd}t� |||�}t
| j |t|�||d�} | j �
| �| j||�d f� | S )NzPool not running�__len__� r r )r� )rR rQ rX r2 r �divmodrq r\ r r� � MapResultrP rO r0 r� r� )
r rB r Zmapperr� r� r� Zextrar� rD r r r
r~ w s&
zPool._map_asyncc C sT t �� }x0|jtks$| jr8|jtkr8| �� t�d� q
W | j �
d � t � d� d S )Ng�������?zworker handler exiting)
r^ �current_threadrR rQ rP � TERMINATEry �time�sleeprO r0 r r7 )�pool�threadr r r
r_ � s zPool._handle_workersc
C sj t �� }x�t| jd �D ]�\}}d }z�x�|D ]�}|jr@t�d� P y ||� W q* tk
r� } zB|d d� \}
} y||
�| d| f� W n t k
r� Y nX W d d } ~ X Y q*X q*W |r�t�d� |r�|d nd} || d � wP W d d } }}
X qW t�d� y:t�d� |�
d � t�d � x|D ]} |d � �q&W W n t k
�rZ t�d
� Y nX t�d � d S ) Nz'task handler found thread._state != RUNr Fzdoing set_length()r r� ztask handler got sentinelz/task handler sending sentinel to result handlerz(task handler sending sentinel to workersz/task handler got OSError when sending sentinelsztask handler exiting) r^ r� �iterr1 rR r r7 r8 �_set�KeyErrorr0 r6 )
� taskqueuer0 r< r� �cacher� ZtaskseqZ
set_lengthr? rE r@ �idx�pr r r
rc � sB
zPool._handle_tasksc C s� t �� }x�y
|� }W n" ttfk
r6 t�d� d S X |jrJt�d� P |d kr^t�d� P |\}}}y|| �||� W n tk
r� Y nX d } }}q
W x�|�r>|jt k�r>y
|� }W n" ttfk
r� t�d� d S X |d k� r�t�d� q�|\}}}y|| �||� W n tk
�r. Y nX d } }}q�W t
| d��r�t�d� y,x&t d�D ]}| j �
� �stP |� �qbW W n ttfk
�r� Y nX t�dt|�|j� d S ) Nz.result handler got EOFError/OSError -- exitingz,result handler found thread._state=TERMINATEzresult handler got sentinelz&result handler ignoring extra sentinelr4 z"ensuring that outqueue is not full�
z7result handler exiting: len(cache)=%s, thread._state=%s)r^ r� r6 r5 r r7 rR r� r� r� r2 rp r4 �pollrq )r< r1 r� r� r? r@ rA �objr r r
rg � sZ
zPool._handle_resultsc c s4 t |�}x&tt�||��}|s"d S | |fV q
W d S )N)r� �tupler �islice)rB �it�sizer� r r r
r� s zPool._get_tasksc C s t d��d S )Nz:pool objects cannot be passed between processes or pickled)�NotImplementedError)r r r r
r" s zPool.__reduce__c C s& t �d� | jtkr"t| _t| j_d S )Nz closing pool)r r7 rR rQ �CLOSEr` )r r r r
r3 s
z
Pool.closec C s$ t �d� t| _t| j_| �� d S )Nzterminating pool)r r7 r� rR r` rl )r r r r
� terminate s
zPool.terminatec C sn t �d� | jtkr td��n| jttfkr4td��| j�� | j �� | j
�� x| j D ] }|�� qZW d S )Nz joining poolzPool is still runningzIn unknown state) r r7 rR rQ rX r� r� r` r rf ri r\ )r r� r r r
r & s
z Pool.joinc C sD t �d� | j�� x*|�� r>| j�� r>| j�� t� d� qW d S )Nz7removing tasks from inqueue until task handler finishedr )
r r7 Z_rlock�acquire�is_aliver4 r� r{ r� r� )r; � task_handlerr� r r r
�_help_stuff_finish2 s
zPool._help_stuff_finishc
C sJ t �d� t|_t|_t �d� | �||t|�� |�� sNt|�dkrNtd��t|_|�d � t �d� t �
� |k r||� � |r�t |d d�r�t �d� x |D ]} | j
d kr�| �� q�W t �d� t �
� |k r�|� � t �d � t �
� |k r�|� � |�rFt |d d��rFt �d
� x0|D ](} | �� �rt �d | j � | � � �qW d S ) Nzfinalizing poolz&helping task handler/workers to finishr z.Cannot have cache with result_hander not alivezjoining worker handlerr� zterminating workerszjoining task handlerzjoining result handlerzjoining pool workerszcleaning up worker %d)r r7 r� rR r� rq r� r. r0 r^ r� r r2 rr r� �pid)
�clsr� r; r< r� Zworker_handlerr� Zresult_handlerr� r� r r r
rj ; s<
zPool._terminate_poolc C s | S )Nr )r r r r
� __enter__k s zPool.__enter__c C s | � � d S )N)r� )r �exc_typeZexc_valZexc_tbr r r
�__exit__n s z
Pool.__exit__)NNr NN)N)N)NNN)r )r )NNN)NNN)"r r r rt rI r rs r] ry rL r} r
r r� r� r� r� r| r� r~ � staticmethodr_ rc rg r� r" r3 r� r r� � classmethodrj r� r� r r r r
r � sD
8
. < 0c @ s@ e Zd Zdd� Zdd� Zdd� Zddd �Zdd
d �Zd d
� ZdS )r� c C s4 t �� | _tt�| _|| _|| _|| _| || j< d S )N) r^ ZEvent�_event�next� job_counterr� rP � _callback�_error_callback)r r� r� r� r r r
r w s
zApplyResult.__init__c C s
| j �� S )N)r� Zis_set)r r r r
�ready s zApplyResult.readyc C s | � � std�| ���| jS )Nz{0!r} not ready)r� rX r/ �_success)r r r r
�
successful� s zApplyResult.successfulNc C s | j �|� d S )N)r� �wait)r �timeoutr r r
r� � s zApplyResult.waitc C s, | � |� | �� st�| jr"| jS | j�d S )N)r� r� r r� �_value)r r� r r r
r1 � s
zApplyResult.getc C sT |\| _ | _| jr$| j r$| �| j� | jr<| j s<| �| j� | j�� | j| j= d S )N)r� r� r� r� r� �setrP r� )r rA r� r r r
r� � s
zApplyResult._set)N)N) r r r r r� r� r� r1 r� r r r r
r� u s
r� c @ s e Zd Zdd� Zdd� ZdS )r� c C sf t j| |||d� d| _d g| | _|| _|dkrLd| _| j�� || j= n|| t || � | _d S )N)r� Tr )
r� r r� r� �
_chunksize� _number_leftr� r� r� �bool)r r� r� �lengthr� r� r r r
r � s
zMapResult.__init__c C s� | j d8 _ |\}}|rp| jrp|| j|| j |d | j �< | j dkr�| jrZ| �| j� | j| j= | j�� nF|s�| jr�d| _|| _| j dkr�| j r�| � | j� | j| j= | j�� d S )Nr r F)
r� r� r� r� r� rP r� r� r� r� )r rA Zsuccess_result�successrD r r r
r� � s"
zMapResult._setN)r r r r r� r r r r
r� � s
r� c @ s: e Zd Zdd� Zdd� Zd dd�ZeZdd � Zd
d � ZdS )
r� c C sJ t �t �� �| _tt�| _|| _t� � | _
d| _ d | _ i | _
| || j< d S )Nr )r^ Z ConditionZLock�_condr� r� r� rP � collections�deque�_items�_index�_length� _unsorted)r r� r r r
r � s
zIMapIterator.__init__c C s | S )Nr )r r r r
�__iter__� s zIMapIterator.__iter__Nc C s� | j �� y| j�� }W nn tk
r� | j| jkr8td �| j �|� y| j�� }W n, tk
r~ | j| jkrttd �td �Y nX Y nX W d Q R X |\}}|r�|S |�d S )N) r� r� �popleft�
IndexErrorr� r� �
StopIterationr� r )r r� r� r� r& r r r
r� � s" zIMapIterator.nextc C s� | j �� | j|krr| j�|� | jd7 _x8| j| jkrd| j�| j�}| j�|� | jd7 _q.W | j �� n
|| j|< | j| jkr�| j| j = W d Q R X d S )Nr )
r� r� r� ru r� �pop�notifyr� rP r� )r rA r� r r r
r� � s
zIMapIterator._setc C s< | j �, || _| j| jkr.| j �� | j| j= W d Q R X d S )N)r� r� r� r� rP r� )r r� r r r
r� s
zIMapIterator._set_length)N) r r r r r� r� �__next__r� r� r r r r
r� � s
r� c @ s e Zd Zdd� ZdS )r� c C sP | j �@ | j�|� | jd7 _| j �� | j| jkrB| j| j= W d Q R X d S )Nr )r� r� ru r� r� r� rP r� )r rA r� r r r
r�
s
zIMapUnorderedIterator._setN)r r r r� r r r r
r� s r� c @ s: e Zd ZdZedd� �Zd dd�Zdd � Zed
d � �ZdS )
r Fc O s ddl m} || |�S )Nr )rI )ZdummyrI )r rC rI r r r
rI s zThreadPool.ProcessNr c C s t �| |||� d S )N)r r )r rm r= r> r r r
r ! s zThreadPool.__init__c C s, t �� | _t �� | _| jj| _| jj| _d S )N)rM rN rk re r0 rd r1 rh )r r r r
rL $ s
zThreadPool._setup_queuesc C sN yx| j dd� qW W n tjk
r, Y nX xt|�D ]}| �d � q8W d S )NF)�block)r1 rM ZEmptyrp r0 )r; r� r� rA r r r
r� * s z ThreadPool._help_stuff_finish)NNr ) r r r rt r� rI r rL r� r r r r
r s
)Nr NF)"�__all__r^ rM r r� rV r� r r r r r rQ r� r� �countr� r r r8 r r r! r$ rG r9 �objectr r� Z AsyncResultr� r� r� r r r r r
�<module>
s@
, f')@
Hacked By AnonymousFox1.0, Coded By AnonymousFox