| Server IP : 162.144.4.212 / Your IP : 216.73.216.108 Web Server : Apache System : Linux gator2125.hostgator.com 5.14.0-162.23.1.9991722448259.nf.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jul 31 18:11:45 UTC 2024 x86_64 User : cozeellc ( 2980) PHP Version : 8.3.31 Disable Function : NONE MySQL : OFF | cURL : ON | WGET : ON | Perl : ON | Python : OFF | Sudo : ON | Pkexec : ON Directory : /usr/libexec/oracle-cloud-agent/plugins/osms/chardet/ |
Upload File : |
a
i��f9! � @ s< d dl mZmZ ddlmZ ddlmZ G dd� de�ZdS )� )�List�Union� )�
CharSetProber)� ProbingStatec s e Zd ZdZdZdZdd�� fdd� Zdd�� fdd � Zee d�d
d ��Z
ee d�d d
��Z e d�dd�Z
e d�dd�Zed�dd�Zed�dd�Zed�dd�Zed�dd�Zee dd�dd �Zee dd �d d �Zeeef ed �d!d"�Z eed�d#d$��Z e d�d%d&�Z � Z S )'�
UTF1632Proberad
This class simply looks for occurrences of zero bytes, and infers
whether the file is UTF16 or UTF32 (low-endian or big-endian)
For instance, files looking like ( [nonzero] )+
have a good probability to be UTF32BE. Files looking like ( [nonzero] )+
may be guessed to be UTF16BE, and inversely for little-endian varieties.
� g�G�z�?N)�returnc sj t � �� d| _dgd | _dgd | _tj| _g d�| _d| _ d| _
d| _ d| _ d| _
d| _| �� d S )Nr � �r r r r F)�super�__init__�position� zeros_at_mod�nonzeros_at_modr � DETECTING�_state�quad�invalid_utf16be�invalid_utf16le�invalid_utf32be�invalid_utf32le�'first_half_surrogate_pair_detected_16be�'first_half_surrogate_pair_detected_16le�reset��self�� __class__� �y/sparta/input/_build_configuration/image_build+validate/lib/bmcenv/lib64/python3.9/site-packages/chardet/utf1632prober.pyr
) s
zUTF1632Prober.__init__c sb t � �� d| _dgd | _dgd | _tj| _d| _d| _ d| _
d| _ d| _ d| _
g d�| _d S )Nr r
Fr )r r r r r r r r r r r r r r r r r r r r 8 s
zUTF1632Prober.resetc C s4 | � � r dS | �� rdS | �� r$dS | �� r0dS dS )Nzutf-32bezutf-32lezutf-16bezutf-16lezutf-16)�is_likely_utf32be�is_likely_utf32le�is_likely_utf16be�is_likely_utf16ler r r r � charset_nameF s zUTF1632Prober.charset_namec C s dS )N� r r r r r �languageS s zUTF1632Prober.languagec C s t d| jd �S )N� �?g @��maxr r r r r �approx_32bit_charsW s z UTF1632Prober.approx_32bit_charsc C s t d| jd �S )Nr( g @r) r r r r �approx_16bit_charsZ s z UTF1632Prober.approx_16bit_charsc C sj | � � }|| jkoh| jd | | jkoh| jd | | jkoh| jd | | jkoh| jd | | jkoh| j S �Nr r � � )r+ �MIN_CHARS_FOR_DETECTIONr �EXPECTED_RATIOr r �r � approx_charsr r r r! ] s
����z UTF1632Prober.is_likely_utf32bec C sj | � � }|| jkoh| jd | | jkoh| jd | | jkoh| jd | | jkoh| jd | | jkoh| j S r- )r+ r0 r r1 r r r2 r r r r" g s
����z UTF1632Prober.is_likely_utf32lec C sV | � � }|| jkoT| jd | jd | | jkoT| jd | jd | | jkoT| j S )Nr r/ r r. )r, r0 r r1 r r r2 r r r r# q s
����z UTF1632Prober.is_likely_utf16bec C sV | � � }|| jkoT| jd | jd | | jkoT| jd | jd | | jkoT| j S )Nr r. r r/ )r, r0 r r1 r r r2 r r r r$ { s
����z UTF1632Prober.is_likely_utf16le)r r c C s� |d dksL|d dksL|d dkrR|d dkrRd|d krHdkrRn nd| _ |d dks�|d dks�|d dkr�|d dkr�d|d kr�dkr�n nd| _d S )
z�
Validate if the quad of bytes is valid UTF-32.
UTF-32 is valid in the range 0x00000000 - 0x0010FFFF
excluding 0x0000D800 - 0x0000DFFF
https://en.wikipedia.org/wiki/UTF-32
r r � �� r. �� Tr/ N)r r )r r r r r �validate_utf32_characters� s4
�
�
�
���
�
�
�
���
z'UTF1632Prober.validate_utf32_characters)�pairr c C s� | j sNd|d kr dkr*n nd| _ qxd|d krBdkrxn qxd| _n*d|d krfdkrrn nd| _ nd| _| js�d|d kr�dkr�n nd| _q�d|d kr�dkr�n q�d| _n*d|d kr�dkr�n nd| _nd| _d S )
a9
Validate if the pair of bytes is valid UTF-16.
UTF-16 is valid in the range 0x0000 - 0xFFFF excluding 0xD800 - 0xFFFF
with an exception for surrogate pairs, which must be in the range
0xD800-0xDBFF followed by 0xDC00-0xDFFF
https://en.wikipedia.org/wiki/UTF-16
r5 r �� T�� r6 Fr N)r r r r )r r8 r r r �validate_utf16_characters� s
z'UTF1632Prober.validate_utf16_characters)�byte_strr c C s� |D ]�}| j d }|| j|< |dkrX| �| j� | �| jdd� � | �| jdd� � |dkrt| j| d7 < n| j| d7 < | j d7 _ q| jS )Nr
r/ r r. r )r r r7 r; r r �state)r r< �c�mod4r r r �feed� s
zUTF1632Prober.feedc C sF | j tjtjhv r| j S | �� dkr.tj| _ n| jdkr@tj| _ | j S )Ng�������?i )r r �NOT_ME�FOUND_IT�get_confidencer r r r r r= � s
zUTF1632Prober.statec C s( | � � s | �� s | �� s | �� r$dS dS )Ng333333�?g )r$ r# r" r! r r r r rC � s �����z UTF1632Prober.get_confidence) �__name__�
__module__� __qualname__�__doc__r0 r1 r
r �property�strr% r' �floatr+ r, �boolr! r" r# r$ r �intr7 r; r �bytes� bytearrayr r@ r= rC �
__classcell__r r r r r s*
r N)�typingr r �
charsetproberr �enumsr r r r r r �<module> s