Hacked By AnonymousFox

Current Path : /opt/cloudlinux/venv/lib64/python3.11/site-packages/pylint/checkers/__pycache__/
Upload File :
Current File : //opt/cloudlinux/venv/lib64/python3.11/site-packages/pylint/checkers/__pycache__/similar.cpython-311.pyc

�

�܋f+����UdZddlmZddlZddlZddlZddlZddlZddlZddl	Z	ddl
Z
ddlmZddl
mZmZmZmZddlmZddlmZmZmZddlmZmZdd	lmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#ddl$Z$dd
l$m%Z%ddl&m'Z'm(Z(m)Z)ddl*m+Z+m,Z,dd
l-m.Z.m/Z/ddl0m1Z1m2Z2erddl3m4Z4dZ5ej6d��Z7ede8��Z9ede8��Z:Gd�de��Z;edee9fZ<ee9dfZ=e#e!eefZ>Gd�d��Z?ede?fZ@Gd�d��ZAGd�d��ZBGd�de��ZCe"de:e:fZDe5fdVd$�ZEdWd(�ZFdXd.�ZGGd/�d0e��ZHGd1�d2��ZI	dYdZd=�ZJejKGd>�d����ZLd?d@iZMdAeNdB<d[dI�ZOGdJ�dKe(eI��ZPd\dN�ZQd]d^dQ�ZRdYd_dT�ZSeTdUkreS��dSdS)`a�A similarities / code duplication command line tool and pylint checker.

The algorithm is based on comparing the hash value of n successive lines of a file.
First the files are read and any line that doesn't fulfill requirement are removed
(comments, docstrings...)

Those stripped lines are stored in the LineSet class which gives access to them.
Then each index of the stripped lines collection is associated with the hash of n
successive entries of the stripped lines starting at the current index (n is the
minimum common lines option).

The common hashes between both linesets are then looked for. If there are matches, then
the match indices in both linesets are stored and associated with the corresponding
couples (start line number/end line number) in both files.

This association is then post-processed to handle the case of successive matches. For
example if the minimum common lines setting is set to four, then the hashes are
computed with four lines. If one of match indices couple (12, 34) is the
successor of another one (11, 33) then it means that there are in fact five lines which
are common.

Once post-processed the values of association table are the result looked for, i.e.
start and end lines numbers of common lines in both files.
�)�annotationsN)�defaultdict)�Callable�	Generator�Iterable�Sequence)�getopt)�BufferedIOBase�BufferedReader�BytesIO)�chain�groupby)
�
TYPE_CHECKING�Any�Dict�List�
NamedTuple�NewType�NoReturn�TextIO�Tuple�Union)�nodes)�BaseChecker�BaseRawFileChecker�table_lines_from_stats)�Section�Table)�MessageDefinitionTuple�Options)�LinterStats�decoding_stream)�PyLinter�z.*\w+�Index�
LineNumberc�$�eZdZUded<ded<dS)�LineSpecifsr&�line_number�str�textN��__name__�
__module__�__qualname__�__annotations__���h/builddir/build/BUILD/cloudlinux-venv-1.0.6/venv/lib/python3.11/site-packages/pylint/checkers/similar.pyr(r(Qs%�����������
�I�I�I�I�Ir2r(�
LinesChunk�SuccessiveLinesLimitsc��eZdZdZdZdd
�ZdS)
�CplSuccessiveLinesLimitsz�Holds a SuccessiveLinesLimits object for each checked file and counts the number
    of common lines between both stripped lines collections extracted from both files.
    ��
first_file�second_file�effective_cmn_lines_nbr9r5r:r;�int�return�Nonec�0�||_||_||_dS�Nr8)�selfr9r:r;s    r3�__init__z!CplSuccessiveLinesLimits.__init__hs!��%���&���&<��#�#�#r2N)r9r5r:r5r;r<r=r>)r-r.r/�__doc__�	__slots__rBr1r2r3r7r7as:��������H�I�=�=�=�=�=�=r2r7�LineSetStartCouplec�>�eZdZdZdZdd�Zdd�Zdd�Zdd�Zdd�Z	dS)r4zlThe LinesChunk object computes and stores the hash of some consecutive stripped
    lines of a lineset.
    ��_fileid�_index�_hash�fileidr*�num_liner<�lines�
Iterable[str]r=r>c�|�||_	t|��|_	td�|D����|_dS)Nc3�4K�|]}t|��V��dSr@)�hash)�.0�lins  r3�	<genexpr>z&LinesChunk.__init__.<locals>.<genexpr>�s(����9�9�C�d�3�i�i�9�9�9�9�9�9r2)rHr%rI�sumrJ)rArKrLrMs    r3rBzLinesChunk.__init__sD��"���Q�"�8�_�_���	��9�9�5�9�9�9�9�9��
�1�1r2�or�boolc�Z�t|t��stS|j|jkSr@)�
isinstancer4�NotImplementedrJ)rArVs  r3�__eq__zLinesChunk.__eq__�s)���!�Z�(�(�	"�!�!��z�Q�W�$�$r2c��|jSr@)rJ�rAs r3�__hash__zLinesChunk.__hash__�s
���z�r2c�8�d|j�d|j�d|j�d�S)Nz<LinesChunk object for file z (z, z)>rGr]s r3�__repr__zLinesChunk.__repr__�s+��X�4�<�X�X�4�;�X�X�$�*�X�X�X�	
r2c�6�d|j�d|j�d|j��S)NzLinesChunk object for file z, starting at line z
 
Hash is rGr]s r3�__str__zLinesChunk.__str__�s7��
$�$�,�
$�
$�4�;�
$�
$��z�
$�
$�	
r2N)rKr*rLr<rMrNr=r>)rVrr=rW�r=r<�r=r*)
r-r.r/rCrDrBr[r^r`rbr1r2r3r4r4xs���������/�I�
2�
2�
2�
2�%�%�%�%�
����
�
�
�
�

�
�
�
�
�
r2c�x�eZdZdZdZdd�Zedd	���Zedd
���Zej	dd���Zdd�Z
dS)r5zA class to handle the numbering of begin and end of successive lines.

    :note: Only the end line number can be updated.
    ��_start�_end�startr&�endr=r>c�"�||_||_dSr@rf)rArirjs   r3rBzSuccessiveLinesLimits.__init__�s��"'��� #��	�	�	r2c��|jSr@)rgr]s r3rizSuccessiveLinesLimits.start�s
���{�r2c��|jSr@�rhr]s r3rjzSuccessiveLinesLimits.end�s
���y�r2�valuec��||_dSr@rn�rAros  r3rjzSuccessiveLinesLimits.end�s
����	�	�	r2r*c�(�d|j�d|j�d�S)Nz<SuccessiveLinesLimits <�;�>>rfr]s r3r`zSuccessiveLinesLimits.__repr__�s��E�$�+�E�E��	�E�E�E�Er2N)rir&rjr&r=r>)r=r&)ror&r=r>rd)r-r.r/rCrDrB�propertyrirj�setterr`r1r2r3r5r5�s���������
#�I�$�$�$�$������X�������X��	�Z�����Z��F�F�F�F�F�Fr2c�H�eZdZUdZded<ded<dd�Zdd�Zdd
�Zdd�ZdS)rEzEIndices in both linesets that mark the beginning of successive lines.r%�fst_lineset_index�snd_lineset_indexr=r*c�(�d|j�d|j�d�S)Nz<LineSetStartCouple <rsrt)rxryr]s r3r`zLineSetStartCouple.__repr__�s#��W�D�$:�W�W�T�=S�W�W�W�	
r2�otherrrWc�z�t|t��stS|j|jko|j|jkSr@)rYrErZrxry�rAr{s  r3r[zLineSetStartCouple.__eq__�sB���%�!3�4�4�	"�!�!��"�e�&=�=�
B��&�%�*A�A�	
r2r<c�T�t|j��t|j��zSr@)rQrxryr]s r3r^zLineSetStartCouple.__hash__�s#���D�*�+�+�d�4�3I�.J�.J�J�Jr2roc�v�tt|j|z��t|j|z����Sr@)rEr%rxryrqs  r3�	incrementzLineSetStartCouple.increment�s:��!��$�(�5�0�1�1��$�(�5�0�1�1�
�
�	
r2Nrd�r{rr=rWrc)ror%r=rE)	r-r.r/rCr0r`r[r^r�r1r2r3rErE�s��������O�O���������
�
�
�
�

�
�
�
�K�K�K�K�
�
�
�
�
�
r2�LineSet�lineset�min_common_linesr<r=�$tuple[HashToIndex_T, IndexToLines_T]c�R��tt��}i}td�|jD������fd�t	|��D��}tt
|���D]�^}}t|j|j��}	|j||zj}n2#t$r%t|jdjdz��}YnwxYwt|��}	t||���||	<t|j
|	g|�R�}
||
�|	����||fS)a`Return two dicts.

    The first associates the hash of successive stripped lines of a lineset
    to the indices of the starting lines.
    The second dict, associates the index of the starting line in the lineset's stripped lines to the
    couple [start, end] lines number in the corresponding file.

    :param lineset: lineset object (i.e the lines in a file)
    :param min_common_lines: number of successive lines that are used to compute the hash
    :return: a dict linking hashes to corresponding start index and a dict that links this
             index to the start and end lines in the file
    c3�$K�|]}|jV��dSr@)r+)rR�xs  r3rTzhash_lineset.<locals>.<genexpr>�s$����9�9�Q�!�&�9�9�9�9�9�9r2c�>��g|]}t�|d�����Sr@)�iter)rR�irMs  �r3�
<listcomp>z hash_lineset.<locals>.<listcomp>�s'���F�F�F��T�%����)�_�_�F�F�Fr2����)rirj)r�list�tuple�stripped_lines�range�	enumerate�zipr&r)�
IndexErrorr%r5r4�name�append)r�r��
hash2index�index2lines�
shifted_linesr��
succ_lines�start_linenumber�end_linenumber�index�l_crMs           @r3�hash_linesetr��s^����T�"�"�J��K�
�9�9�'�"8�9�9�9�9�9�E�F�F�F�F�e�4D�.E�.E�F�F�F�M�#�C��$7�8�8�
&�
&���J�%�g�&<�Q�&?�&K�L�L��	T�$�3�A�8H�4H�I�U�N�N���	T�	T�	T�'��(>�r�(B�(N�QR�(R�S�S�N�N�N�	T�����a����2�"��
�
�
��E�����u�:�z�:�:�:���3����u�%�%�%�%��{�"�"s�B!�!,C�C�all_couples�CplIndexToCplLines_Tr>c��t|�����D]�}g}|�td����}||vr�||jj||j_||jj||j_||xjdz
c_|�|��|�td����}||v��|D](}	|�	|���#t$rY�%wxYw��dS)aRemoves all successive entries in the dictionary in argument.

    :param all_couples: collection that has to be cleaned up from successive entries.
                        The keys are couples of indices that mark the beginning of common entries
                        in both linesets. The values have two parts. The first one is the couple
                        of starting and ending line numbers of common successive lines in the first file.
                        The second part is the same for the second file.

    For example consider the following dict:

    >>> all_couples
    {(11, 34): ([5, 9], [27, 31]),
     (23, 79): ([15, 19], [45, 49]),
     (12, 35): ([6, 10], [28, 32])}

    There are two successive keys (11, 34) and (12, 35).
    It means there are two consecutive similar chunks of lines in both files.
    Thus remove last entry and update the last line numbers in the first entry

    >>> remove_successive(all_couples)
    >>> all_couples
    {(11, 34): ([5, 10], [27, 32]),
     (23, 79): ([15, 19], [45, 49])}
    r�N)r��keysr�r%r9rjr:r;r��pop�KeyError)r��couple�	to_remove�test�targets     r3�remove_successiver�s1��4��(�(�*�*�+�+�����	�����a���)�)���k�!�!�1<�T�1B�1M�1Q�K���*�.�2=�d�2C�2O�2S�K���+�/����6�6�!�;�6�6����T�"�"�"��>�>�%��(�(�+�+�D��k�!�!� �	�	�F�
�����'�'�'�'���
�
�
���
����	��s�&C<�<
D	�D	�ls_1�	stindex_1�ls_2�	stindex_2�common_lines_nbc��d�|j|||z�D��}d�|j|||z�D��}td�t||��D����S)a|Return the effective number of common lines between lineset1
    and lineset2 filtered from non code lines.

    That is to say the number of common successive stripped
    lines except those that do not contain code (for example
    a line with only an ending parenthesis)

    :param ls_1: first lineset
    :param stindex_1: first lineset starting index
    :param ls_2: second lineset
    :param stindex_2: second lineset starting index
    :param common_lines_nb: number of common successive stripped lines before being filtered from non code lines
    :return: the number of common successive stripped lines that contain code
    c�Z�g|](}t�|j���!|j��)Sr1��REGEX_FOR_LINES_WITH_CONTENT�matchr+�rR�lspecifs  r3r�z(filter_noncode_lines.<locals>.<listcomp>D�@������'�-�-�g�l�;�;������r2c�Z�g|](}t�|j���!|j��)Sr1r�r�s  r3r�z(filter_noncode_lines.<locals>.<listcomp>Ir�r2c3�(K�|]
\}}||kV��dSr@r1)rR�sline_1�sline_2s   r3rTz'filter_noncode_lines.<locals>.<genexpr>Ns,����X�X�&6�g�w�w�'�!�X�X�X�X�X�Xr2)r�rUr�)r�r�r�r�r��stripped_l1�stripped_l2s       r3�filter_noncode_linesr�/s���*���*�9�y�?�7R�+R�S����K�
���*�9�y�?�7R�+R�S����K�
�X�X�#�k�;�:W�:W�X�X�X�X�X�Xr2c�V�eZdZUded<ded<ded<ded<ded<ded	<ded
<dS)�Commonalityr<�cmn_lines_nbr��fst_lsetr&�fst_file_start�fst_file_end�snd_lset�snd_file_start�snd_file_endNr,r1r2r3r�r�Qsf������������������������������������r2r�c�r�eZdZdZeddddfd&d�Z	d'd(d�Zd)d�Zd*d�Zd+d�Z	d,d�Z
d-d�Zd.d �Zd/d"�Z
d0d%�Zd
S)1�Similarz-Finds copy-pasted lines of code in a project.F�	min_linesr<�ignore_commentsrW�ignore_docstrings�ignore_imports�ignore_signaturesr=r>c�
�t|t��r|jj|_ntj��|_||j_||j_||j_	||j_
||j_g|_dSr@)
rYr�linter�config�	namespace�argparse�	Namespace�min_similarity_linesr�r�r�r��linesets)rAr�r�r�r�r�s      r3rBzSimilar.__init__^sq���d�K�(�(�	2�!�[�/�D�N�N�%�/�1�1�D�N�.7���+�)8���&�+<���(�(6���%�+<���(�')��
�
�
r2N�streamidr*�stream�STREAM_TYPES�encoding�
str | Nonec��t|t��r|�t�t||��j}n|j}	|��}n#t
$rg}YnwxYw|j�t|||j	j
|j	j|j	j|j	j
t|d��r|jjnd�����dS)z)Append a file to search for similarities.Nr���line_enabled_callback)rYr
�
ValueErrorr"�	readlines�UnicodeDecodeErrorr�r�r�r�r�r�r�r��hasattrr��_is_one_message_enabled)rAr�r�r�r�rMs      r3�
append_streamzSimilar.append_streamss����f�n�-�-�	)��� � �'���9�9�C�I�I��(�I�	��I�K�K�E�E��!�	�	�	��E�E�E�	����	
�
��������.���0���-���0��4��*�*�'�d�k�&I�&I��

�

�

�	
�	
�	
�	
�	
s�
A�A�Ac�x�|jjdkrdS|�|�����dS)z=Start looking for similarities and display results on stdout.rN)r�r��
_display_sims�
_compute_simsr]s r3�runzSimilar.run�s=���>�.�!�3�3��F����4�-�-�/�/�0�0�0�0�0r2�)list[tuple[int, set[LinesChunkLimits_T]]]c���tt��}|���D]m}|j}|j}|j}|j}|j}|j}|j	}	||}
|
D]}|||f|vs|||	f|vrn�|
�
|||f|||	fh���ng}|���D]!\}}
|
D]}|�
||f����"|���|�
��|S)z'Compute similarities in appended files.)rr��
_iter_simsr�r�r�r�r�r�r�r��items�sort�reverse)rA�
no_duplicates�commonality�num�lineset1�start_line_1�
end_line_1�lineset2�start_line_2�
end_line_2�	duplicate�couples�sims�	ensembles�cplss               r3r�zSimilar._compute_sims�s^��BM�d�BS�BS�
��?�?�,�,�	�	�K��*�C�"�+�H�&�5�L�$�1�J�"�+�H�&�5�L�$�1�J�%�c�*�I�$�

�

���l�J�7�7�B�B�� ��G��	G�G�
�E�G�� � �!�<��<�!�<��<������;=��+�1�1�3�3�	)�	)�N�C��!�
)�
)�����S�$�K�(�(�(�(�
)��	�	�����������r2�similaritiesc�N�|�|��}t|��dS)z(Display computed similarities on stdout.N)�_get_similarity_report�print)rAr��reports   r3r�zSimilar._display_sims�s&���,�,�\�:�:��
�f�
�
�
�
�
r2c
���d}d}|D]�\}}|d|�dt|���d�z
}t|��}dx}x}}	|D]\}}}	|d|j�d|�d	|	�d
�z
}�|rC|j||	�D]3}
||
���rd|
����d�ndz
}�4||t|��dz
zz
}��td
�|jD����}|d|�d|�d|dz|zd�d�z
}|S)z"Create a report from similarities.�r�
z similar lines in z files
N�==�:[�:z]
z   r�c3�4K�|]}t|��V��dSr@��len�rRr�s  r3rTz1Similar._get_similarity_report.<locals>.<genexpr>�s(����$O�$O�g�S��\�\�$O�$O�$O�$O�$O�$Or2zTOTAL lines=z duplicates=z	 percent=�Y@z.2f)r�sortedr��_real_lines�rstriprUr�)rAr�r�duplicated_line_number�numberr��	couples_l�line_set�
start_line�end_line�line�total_line_numbers            r3rzSimilar._get_similarity_report�s�����&'��+�		D�		D�O�F�G��K�6�K�K�S��\�\�K�K�K�K�F��w���I�/3�3�H�3�z�H�2;�
K�
K�.��*�h��J�x�}�J�J�
�J�J�X�J�J�J�J����
Q�$�0��H�1D�E�Q�Q�D�������P�5�D�K�K�M�M�5�5�5�5�D�P�F�F�"�f��I����0B�&C�C�"�"�!$�$O�$O���$O�$O�$O�!O�!O���
R�,�
R�
R�0�
R�
R�-��5�8I�I�Q�
R�
R�
R�	
��
�
r2r�r�r��"Generator[Commonality, None, None]c
#���K�t||jj��\�}t||jj��\}}t������}t|�����}t||z�fd����}i}	t|t
jd�����D]�}
tj	�|
||
��D]o}|d}|d}
ttj||��tj||
��|jj���|	t||
��<�p��t|	��|	���D]\}}|j}|j}|j}t%|||jj|jj||jj|jj���}t/|||||��}||jjkr|V���dS)	atFind similarities in the two given linesets.

        This the core of the algorithm. The idea is to compute the hashes of a
        minimal number of successive lines of each lineset and then compare the
        hashes. Every match of such comparison is stored in a dict that links the
        couple of starting indices in both linesets to the couple of corresponding
        starting and ending lines in both files.

        Last regroups all successive couples in a bigger one. It allows to take into
        account common chunk of lines that have more than the minimal number of
        successive lines required.
        c� ���|dS�Nrr1)�m�hash_to_index_1s �r3�<lambda>z&Similar._find_common.<locals>.<lambda>�s���?�1�+=�a�+@�r2��keyrIrr�)r;)r�r�r�r�r�r�r�N)r�r�r��	frozensetr�r�operator�
attrgetter�	itertools�productr7�copyrEr�r�rxryr;r�r9rirjr:r�)rAr�r��index_to_lines_1�hash_to_index_2�index_to_lines_2�hash_1�hash_2�
common_hashesr��c_hash�indices_in_linesets�index_1�index_2�cml_stripped_l�cmn_l�
start_index_1�
start_index_2�nb_common_lines�com�
eff_cmn_nbrs                     @r3�_find_commonzSimilar._find_common�sB�����&-9��d�n�9�-
�-
�)��)�-9��d�n�9�-
�-
�)��)�)2�/�2F�2F�2H�2H�(I�(I��(1�/�2F�2F�2H�2H�(I�(I��.4��V�O�!@�!@�!@�!@�/
�/
�/
�
�-/���]��0C�H�0M�0M�N�N�N�	�	�F�'0�'8���'���)@�(�(�
�
�#�.�a�0��-�a�0��-��I�.�w�7�8�8��I�.�w�7�8�8�+/�>�+N�����&�w��8�8���
�	�+�&�&�&�%0�%6�%6�%8�%8�	�	�!�N�E�*�<�M�*�<�M�#�:�O��,�!�$�/�5�"�-�1�!�$�0�6�"�.�2����C�.��-��=�/���J��D�N�?�?�?��	�	�	��)	�	r2c#�K�t|jdd���D]6\}}|j|dzd�D]}|�||��Ed{V����7dS)zXIterate on similarities among all files, by making a Cartesian
        product.
        Nr�r�)r�r�r9)rA�idxr�r�s    r3r�zSimilar._iter_sims*s�����&�d�m�C�R�C�&8�9�9�	@�	@�L�C�� �M�#��'�)�)�4�
@�
@���,�,�W�h�?�?�?�?�?�?�?�?�?�?�
@�	@�	@r2�
list[LineSet]c��|jS)z�Returns the data we can use for a map/reduce process.

        In this case we are returning this instance's Linesets, that is all file
        information that will later be used for vectorisation.
        �r�r]s r3�get_map_datazSimilar.get_map_data2s���}�r2�linesets_collection�list[list[LineSet]]c�(�d�|D��|_dS)�yReduces and recombines data into a format that we can report on.

        The partner function of get_map_data()
        c��g|]	}|D]}|���
Sr1r1)rRr�rs   r3r�z2Similar.combine_mapreduce_data.<locals>.<listcomp>?s%��U�U�U�'�W�U�U�T��U�U�U�Ur2Nr>)rAr@s  r3�combine_mapreduce_datazSimilar.combine_mapreduce_data:s��
V�U�-@�U�U�U��
�
�
r2)r�r<r�rWr�rWr�rWr�rWr=r>r@)r�r*r�r�r�r�r=r>�r=r>)r=r�)r�r�r=r>)r�r�r=r*)r�r�r�r�r=r)r=r�r=r<)r@rAr=r>)r-r.r/rC�DEFAULT_MIN_SIMILARITY_LINErBr�r�r�r�rr9r�r?rEr1r2r3r�r�[s������7�7�5� %�"'�$�"'�
*�*�*�*�*�,KO�
�
�
�
�
�>1�1�1�1�%�%�%�%�N��������2I�I�I�I�V@�@�@�@�����V�V�V�V�V�Vr2r�rMrNr�rWr�r�r�r��!Callable[[str, int], bool] | None�list[LineSpecifs]c
����|s|r'tjd�|����}|r/d�|jD��}d�t	|d����D��}d}	|r2d�fd���g|��}
tt
d
�|
D�����}g}d}
t|d���D�]O\}}|�
|d|��s�|���}|r�|
s}|�	d��s|�	d��r|dd�}
|dd�}n>|�	d��s|�	d��r|dd�}
|dd�}|
r|�
|
��rd}
d}|r|�||	��}	|	rd}|r.|�dd��d���}|r||vrd}|r4|�
t|t|dz
���������Q|S)a�Return tuples of line/line number/line type with leading/trailing white-space and
    any ignored code features removed.

    :param lines: a collection of lines
    :param ignore_comments: if true, any comment in the lines collection is removed from the result
    :param ignore_docstrings: if true, any line that is a docstring is removed from the result
    :param ignore_imports: if true, any line that is an import is removed from the result
    :param ignore_signatures: if true, any line that is part of a function signature is removed from the result
    :param line_enabled_callback: If called with "R0801" and a line number, a return value of False will disregard
           the line
    :return: the collection of line/line number/line type tuples
    rc3�pK�|]1}|jt|tjtjf��fV��2dSr@)�linenorYr�Import�
ImportFrom)rR�nodes  r3rTz!stripped_lines.<locals>.<genexpr>YsQ����$
�$
���[�*�T�E�L�%�:J�+K�L�L�M�$
�$
�$
�$
�$
�$
r2c�H�i|]\}}|td�|D������ S)c3� K�|]	\}}|V��
dSr@r1)rR�_�	is_imports   r3rTz,stripped_lines.<locals>.<dictcomp>.<genexpr>^s&����K�K�l�a��	�K�K�K�K�K�Kr2)�all)rRrM�node_is_import_groups   r3�
<dictcomp>z"stripped_lines.<locals>.<dictcomp>]sG��
�
�
�,��,�
�C�K�K�6J�K�K�K�K�K�
�
�
r2c��|dSrr1)r�s r3rz stripped_lines.<locals>.<lambda>`s
���!��r2r F�	functions�list[nodes.NodeNG]�tree�nodes.NodeNGr=c���|jD]z}t|tjtjf��r|�|��t|tjtjtjf��r�||���{|S)zeRecursively get all functions including nested in the classes from the
            tree.
            )�bodyrYr�FunctionDef�AsyncFunctionDefr��ClassDef)rYr[rP�_get_functionss   �r3rbz&stripped_lines.<locals>._get_functionsfs�����	�
4�
4���d�U�%6��8N�$O�P�P�+��$�$�T�*�*�*����^�U�%6��8N�O���4�#�N�9�d�3�3�3���r2c3�K�|]:}t|j|jr|jdjn	|jdz��V��;dS)rr�N)r�rMr^�tolineno)rR�funcs  r3rTz!stripped_lines.<locals>.<genexpr>|sf������
�	���/3�y�O��	�!��+�+�d�m�a�>O��������r2Nr�)ri�R0801z"""z'''�zr"""zr'''r$�#r)r+r))rYrZr[r\r=rZ)�astroid�parse�joinr^r�setr
r��strip�
startswith�endswith�get�splitr�r(r&)rMr�r�r�r�r�r[�node_is_import_by_lineno�line_begins_import�current_line_is_importrY�signature_lines�
strippedlines�	docstringrMrrbs                @r3r�r�Bs����(�-�*�-��}�R�W�W�U�^�^�,�,���'�$
�$
��	�$
�$
�$
� �
�
�07�(�n�n�1�1�1�
�
�
��"'��� 
�	�	�	�	�	�	�&#�N�2�t�,�,�	�����
!*����
�

�

���M��I�!�%�q�1�1�1������ �,�5J�5J��V�6
�6
�,�
��z�z�|�|���	��
$��?�?�5�)�)�$�T�_�_�U�-C�-C�$� $�R�a�R��I�����8�D�D��_�_�V�,�,�$�����0G�0G�$� $�Q�q�S�	�I�����8�D��
��=�=��+�+�%� $�I����	�%7�%;�%;��.�&�&�"�&�
����	1��:�:�c�1�%�%�a�(�.�.�0�0�D��	��?�!:�!:��D��	�� � ���:�f�q�j�3I�3I�J�J�J�
�
�
���r2c��eZdZdZ					d d!d�Zd"d�Zd#d�Zd$d�Zd%d�Zd#d�Z	d&d�Z
ed'd���Zed(d���Z
dS))r�z�Holds and indexes all the lines of a single source file.

    Allows for correspondence between real lines of the source file and stripped ones, which
    are the real ones from which undesired patterns have been removed.
    FNr�r*rM�	list[str]r�rWr�r�r�r�rIr=r>c�V�||_||_t||||||���|_dS)Nr�)r�rr��_stripped_lines)rAr�rMr�r�r�r�r�s        r3rBzLineSet.__init__�sB����	� ���-������"7�
 
� 
� 
����r2c��d|j�d�S)Nz
<Lineset for �>�r�r]s r3rbzLineSet.__str__�s��+�t�y�+�+�+�+r2r<c�*�t|j��Sr@)rrr]s r3�__len__zLineSet.__len__�s���4�#�$�$�$r2r�r(c��|j|Sr@�r{)rAr�s  r3�__getitem__zLineSet.__getitem__�s���#�E�*�*r2r{c�"�|j|jkSr@r~r}s  r3�__lt__zLineSet.__lt__�s���y�5�:�%�%r2c� �t|��Sr@)�idr]s r3r^zLineSet.__hash__�s
���$�x�x�r2rc�P�t|t��sdS|j|jkS)NF)rYr��__dict__r}s  r3r[zLineSet.__eq__�s(���%��)�)�	��5��}���.�.r2rJc��|jSr@r�r]s r3r�zLineSet.stripped_lines�s���#�#r2c��|jSr@)rr]s r3�
real_lineszLineSet.real_lines�s����r2)FFFFN)r�r*rMryr�rWr�rWr�rWr�rWr�rIr=r>rdrc)r�r<r=r()r{r�r=rWr�)r=rJ)r=ry)r-r.r/rCrBrbr�r�r�r^r[rur�r�r1r2r3r�r��s��������!&�"'�$�"'�CG�
�
�
�
�
�*,�,�,�,�%�%�%�%�+�+�+�+�&�&�&�&�����/�/�/�/�
�$�$�$��X�$�� � � ��X� � � r2rf)zSimilar lines in %s files
%szduplicate-codez�Indicates that a set of similar lines has been detected among multiple file. This usually means that the code should be refactored to avoid this duplication.z!dict[str, MessageDefinitionTuple]�MSGS�sectr�statsr!�	old_stats�LinterStats | Nonec��gd�}|t||d��z
}|�t|ddd�����dS)z0Make a layout with some stats about duplication.)r�now�previous�
difference�duplicated_linesr$r�)�children�cols�rheaders�cheadersN)rr�r)r�r�r�rMs    r3�report_similaritiesr��sP��
2�1�1�E�	�
#�E�9�6H�
I�
I�I�E��K�K��u�1�q�1�E�E�E�F�F�F�F�Fr2c
��eZdZUdZdZeZdedddd�fdd	d
ddd�fd
d	d
ddd�fdd	d
ddd�fdd	d
ddd�ffZde	d<dde
ffZd'd�Zd(d�Z
d)d�Zd(d �Zd*d"�Zd+d%�Zd&S),�SimilarCheckerz�Checks for similarities and duplicated code.

    This computation may be memory / CPU intensive, so you
    should disable it if you experience some problems.
    r�zmin-similarity-linesr<z<int>z%Minimum lines number of a similarity.)�default�type�metavar�help�ignore-commentsT�ynz<y or n>z4Comments are removed from the similarity computation�ignore-docstringsz6Docstrings are removed from the similarity computation�ignore-importsz3Imports are removed from the similarity computation�ignore-signaturesz6Signatures are removed from the similarity computationr �options�RP0801�Duplicationr�r#r=r>c��tj||��t�||jjj|jjj|jjj|jjj|jjj	���dS)N)r�r�r�r�r�)
rrBr�r�r�r�r�r�r�r�)rAr�s  r3rBzSimilarChecker.__init__;sv���#�D�&�1�1�1������k�(�=� �K�.�>�"�k�0�B��;�-�<�"�k�0�B�
	�	
�	
�	
�	
�	
r2c�P�g|_|jj���dS)z=Init the checkers: reset linesets and statistics information.N)r�r�r��reset_duplicated_linesr]s r3�openzSimilarChecker.openFs&����
����0�0�2�2�2�2�2r2rP�nodes.Modulec��|jj�tjdt��|���5}|�|jj||j��ddd��dS#1swxYwYdS)z�Process a module.

        the module's content is accessible via the stream object

        stream must implement the readlines method
        Nz�In pylint 3.0 the current_name attribute of the linter object should be a string. If unknown it should be initialized as an empty string.)r��current_name�warnings�warn�DeprecationWarningr�r��
file_encoding)rArPr�s   r3�process_modulezSimilarChecker.process_moduleKs����;�#�+��M�N�#�
�
�
��[�[�]�]�	U�f����t�{�7���AS�T�T�T�	U�	U�	U�	U�	U�	U�	U�	U�	U�	U�	U�	U����	U�	U�	U�	U�	U�	Us�'A/�/A3�6A3c��td�|jD����}d}|jj}|���D]�\}}g}dx}x}}	|D]*\}}}	|�d|j�d|�d|	�d����+|���|r9|j||	�D])}
|�|
�	�����*|�
dt|��d	�|��f�
��||t|��dz
zz
}��|xj
t|��z
c_
|xjt!|o|dz|z��z
c_dS)
zBCompute and display similarities on closing (i.e. end of parsing).c3�4K�|]}t|��V��dSr@r
rs  r3rTz'SimilarChecker.close.<locals>.<genexpr>_s(����>�>�W�C��L�L�>�>�>�>�>�>r2rNrrr�]rfr)�argsr�r
)rUr�r�r�r�r�r�r�r�r�add_messagerrk�nb_duplicated_linesr<�percent_duplicated_lines�float)rA�total�
duplicatedr�r�r��msgr�rrrs           r3�closezSimilarChecker.close]s����>�>��
�>�>�>�>�>���
���!�� �.�.�0�0�	3�	3�L�C���C�.2�2�G�2�j�8�18�
J�
J�-���X��
�
�H���H�H�
�H�H�X�H�H�H�I�I�I�I��H�H�J�J�J��
.�#�.�z�(�/B�C�.�.�D��J�J�t�{�{�}�}�-�-�-�-����W�C��L�L�$�)�)�C�.�.�+I��J�J�J��#��W����!1�2�2�J�J�
�!�!�S��_�_�4�!�!�
�&�&�%��0T�*�u�:L�u�:T�*U�*U�U�&�&�&�&r2r<c�6�t�|��S)zPassthru override.)r�r?r]s r3r?zSimilarChecker.get_map_datars���#�#�D�)�)�)r2�datarAc�>�t�||���dS)rC)r@N)r�rE)rAr�r�s   r3�reduce_map_datazSimilarChecker.reduce_map_datavs#��
	�&�&�t��&�F�F�F�F�Fr2N�r�r#r=r>rF)rPr�r=r>rG)r�r#r�rAr=r>)r-r.r/rCr�r��msgsrHr�r0r��reportsrBr�r�r�r?r�r1r2r3r�r��s�����������D��D�

#�6��"�?�	
�
�	
�
���%�N�	
�
�	
�
 ���%�P�	
�
�	
�
���%�M�	
�
�	
�
 ���%�P�	
�
�	
�K.�G�.�.�.�.�`�-�)<�=�?�G�	
�	
�	
�	
�3�3�3�3�
U�U�U�U�$V�V�V�V�**�*�*�*�G�G�G�G�G�Gr2r�r�r#c�J�|�t|����dSr@)�register_checkerr�)r�s r3�registerr�~s$��
���N�6�2�2�3�3�3�3�3r2�statusrc��td��t��td��tj|��dS)z'Display command line usage information.z*finds copy pasted blocks in a set of filesz�Usage: symilar [-d|--duplicates min_duplicated_lines] [-i|--ignore-comments] [--ignore-docstrings] [--ignore-imports] [--ignore-signatures] file1...N)r�sys�exit)r�s r3�usager��sJ��	�
6�7�7�7�	�G�G�G�	�	`�����H�V�����r2�argv�Sequence[str] | Nonec�R�|�tjdd�}d}gd�}t}d}d}d}d}tt	|��||��\}}	|D]G\}
}|
dvrt|��}�|
dvrt
���,|
dvrd	}�3|
d
vrd	}�:|
dvrd	}�A|
dvrd	}�H|	st
d��t|||||��}|	D]A}
t|
d
���5}|�	|
|��ddd��n#1swxYwY�B|�
��tjd��dS)z%Standalone command line access point.Nr��hdi)r�zduplicates=r�r�r�r�F>�-d�--duplicates>�-h�--help>�-i�--ignore-commentsT>�--ignore-docstrings>�--ignore-imports>�--ignore-signatureszutf-8)r�r)r�r�rHr	r�r<r�r�r�r�r�r�)r��s_opts�l_optsr�r�r�r�r��optsr��opt�val�sim�filenamer�s               r3�Runr��s����|��x����|��
�F����F�,�I��O����N�����T�
�
�F�F�3�3�J�D�$��%�%���S��(�(�(��C���I�I�
�$�
$�
$��G�G�G�G�
�/�
/�
/�"�O�O�
�+�
+�
+� $���
�(�
(�
(�!�N�N�
�+�
+�
+� $�����
�a����
��?�$5�~�GX���C��0�0��
�(�W�
-�
-�
-�	0�����h��/�/�/�	0�	0�	0�	0�	0�	0�	0�	0�	0�	0�	0����	0�	0�	0�	0���G�G�I�I�I��H�Q�K�K�K�K�Ks�C3�3C7	�:C7	�__main__)r�r�r�r<r=r�)r�r�r=r>)r�r�r�r%r�r�r�r%r�r<r=r<r@)rMrNr�rWr�rWr�rWr�rWr�rIr=rJ)r�rr�r!r�r�r=r>r�)r)r�r<r=r)r�r�r=r)UrC�
__future__rr�r'�	functoolsr%r#�rer�r��collectionsr�collections.abcrrrrr	�ior
rrr
r�typingrrrrrrrrrrrir�pylint.checkersrrr�pylint.reporters.ureports.nodesrr�
pylint.typingrr �pylint.utilsr!r"�pylint.lintr#rH�compiler�r<r%r&r(�
HashToIndex_T�IndexToLines_Tr�r7r�r4r5rE�LinesChunkLimits_Tr�r�r�r�r�r��total_orderingr�r�r0r�r�r�r�r�r-r1r2r3�<module>r�st��
���2#�"�"�"�"�"���������������������	�	�	�	�
�
�
�
�����#�#�#�#�#�#�C�C�C�C�C�C�C�C�C�C�C�C�������6�6�6�6�6�6�6�6�6�6�$�$�$�$�$�$�$�$�����������������������������������S�S�S�S�S�S�S�S�S�S�:�:�:�:�:�:�:�:�9�9�9�9�9�9�9�9�5�5�5�5�5�5�5�5��%�$�$�$�$�$�$���)�r�z�(�3�3��	��������W�\�3�
'�
'�
������*�����\�4��;�.�/�
��e�4�4�5���V�^�W�4�5��=�=�=�=�=�=�=�=�(�0�2J�J�K��$
�$
�$
�$
�$
�$
�$
�$
�NF�F�F�F�F�F�F�F�8
�
�
�
�
��
�
�
�:�9�j�*�<�=��/J�&#�&#�&#�&#�&#�R(�(�(�(�VY�Y�Y�Y�D�����*����dV�dV�dV�dV�dV�dV�dV�dV�Z@D�
f�f�f�f�f�R��6 �6 �6 �6 �6 �6 �6 ���6 �t��+������G�G�G�G�G�G�G�G�G�'��G�G�G�D4�4�4�4������*�*�*�*�*�Z�z����C�E�E�E�E�E��r2

Hacked By AnonymousFox1.0, Coded By AnonymousFox