| Server IP : 162.144.4.212 / Your IP : 216.73.216.108 Web Server : Apache System : Linux gator2125.hostgator.com 5.14.0-162.23.1.9991722448259.nf.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Jul 31 18:11:45 UTC 2024 x86_64 User : cozeellc ( 2980) PHP Version : 8.3.31 Disable Function : NONE MySQL : OFF | cURL : ON | WGET : ON | Perl : ON | Python : OFF | Sudo : ON | Pkexec : ON Directory : /lib64/python3.9/site-packages/hgext/lfs/__pycache__/ |
Upload File : |
a
�+�bim � @ s� d dl mZ d dlZd dlZd dlZd dlZd dlZd dlZd dlZd dl m
Z
d dl m Z d dl
mZ d dlmZmZmZmZmZmZmZmZmZ d dl m Z m Z dd l m Z e�!d
�Z"G d d � d ej�Z#G d
d� de#�Z$G dd� dej%�Z&G dd� de'�Z(dd� Z)G dd� dej*j+�Z,G dd� de'�Z-G dd� de'�Z.G dd � d e'�Z/G d d � d e'�Z0e-e-e.e/e0d �Z1d d!� Z2d"d#� Z3d*d$d%�Z4G d&d'� d'ej5�Z6G d(d)� d)ej7�Z8dS )+� )�absolute_importN)�_)�getattr)�hex) �encoding�error�httpconnection�pathutil�pycompat�url�util�vfs�worker)�
stringutil�urlutil� )�lfutils \A[a-f0-9]{64}\Zc s& e Zd Z� fdd�Zddd�Z� ZS )�lfsvfsc s: t �|�st�d| ��tt| ��|dd� |dd� �S )z9split the path at first two characters, like: XX/XXXXX...s unexpected lfs path: %sr r N)�_lfsre�matchr �ProgrammingError�superr �join)�self�path�� __class__� �9/usr/lib64/python3.9/site-packages/hgext/lfs/blobstore.pyr . s
z lfsvfs.joinNc # s� t j�| j�}tt�|��}g }t j| �| j|p2d�|d�D ]:\� }}� |d� � t� �dkr<|� � fdd�|D �� q<dg |fV dS )z�Yield (dirpath, [], oids) tuple for blobs under path
Oids only exist in the root of this vfs, so dirpath is always ''.
� )�onerrorNr c s" g | ]}t �� | �r� | �qS r )r r )�.0�f��dirpathr r �
<listcomp>G r z lfsvfs.walk.<locals>.<listcomp>)
�osr �normpath�base�lenr Z normasprefix�walkZreljoin�extend)r r r �rootZ prefixlen�oids�dirs�filesr r# r r* 4 s � �z lfsvfs.walk)NN)�__name__�
__module__� __qualname__r r* �
__classcell__r r r r r - s r c @ s6 e Zd Zdd� Zdd� Zdd� Zd dd �Zd
d � ZdS )
�nullvfsc C s d S �Nr �r r r r �__init__N s znullvfs.__init__c C s dS )NFr �r �oidr r r �existsQ s znullvfs.existsc C s t tjt�d| ���d S )Ns %s: No such file or directory)�IOError�errno�ENOENTr
Zsysstrr8 r r r �readT s �z nullvfs.readNc C s
dg g fS )Nr r )r r r r r r r* ^ s z nullvfs.walkc C s d S r5 r )r r9 �datar r r �writea s z
nullvfs.write)NN)r0 r1 r2 r7 r: r> r* r@ r r r r r4 M s
r4 c s( e Zd ZdZ� fdd�Zdd� Z� ZS )�
lfsuploadfilez+a file-like object that supports keepalive.c s" t t| ��||d� | jj| _d S �Ns rb)r rA r7 �_datar> )r �ui�filenamer r r r7 h s zlfsuploadfile.__init__c C s d S r5 r r6 r r r �
_makeprogressl s zlfsuploadfile._makeprogress)r0 r1 r2 �__doc__r7 rF r3 r r r r rA e s rA c @ sj e Zd ZdZdd� Zdd� Zdd� Zdd � Zd
d � Zd d
� Z dd� Z
ddd�Z dd� Z dd� Z
dd� ZdS )�localz�Local blobstore for large file contents.
This blobstore is used both as a cache and as a staging area for large blobs
to be uploaded to the remote blobstore.
c C sR |j �d�}t|�| _|j�dd�r.t� | _nt� |jd�}t|�| _|j| _d S )Ns lfs/objects� experimentals lfs.disableusercache� lfs)
Zsvfsr r r
rD �
configboolr4 �cachevfsr Z
_usercachedir)r �repo�fullpathZ usercacher r r r7 w s
zlocal.__init__c C s t | �|�d�S )zgOpen a read-only file descriptor to the named blob, in either the
usercache or the local store.�rb)�openr r8 r r r rP � s z
local.openc C s$ | j �|�r| j �|�S | j�|�S )a{ Build the path for the given blob ``oid``.
If the blob exists locally, the path may point to either the usercache
or the local store. If it doesn't, it will point to the local store.
This is meant for situations where existing code that isn't LFS aware
needs to open a blob. Generally, prefer the ``open`` method on this
class.
)rL r: r r
r8 r r r r � s z
local.pathc
C s� t �� }d}| j|ddd���}tj|dd�D ]$}|�|� |�|� |t|�7 }q,|dur�t|�|kr�d}t t
|�|t|�f ��t |� � �} | |kr�t
t
d �| ��W d � n1 s�0 Y | �|� dS )
znRead the blob from the remote source in chunks, verify the content,
and write to this local blobstore.r � wbT�Z
atomictemp� ��sizeNsX Response length (%s) does not match Content-Length header (%d): likely server-side crashs corrupt remote lfs object: %s)�hashlib�sha256r
r �
filechunkiterr@ �updater) �int�LfsRemoteErrorr r �digest�LfsCorruptionError�_linktousercache)
r r9 �srcZcontent_lengthrW rU �fp�chunk�msg�realoidr r r �download� s"
�
�"zlocal.downloadc C sH | j |ddd��}|�|� W d � n1 s00 Y | �|� dS )z�Write blob to local blobstore.
This should only be called from the filelog during a commit or similar.
As such, there is no need to verify the data. Imports from a remote
store must use ``download()`` instead.rQ TrR N)r
r@ r^ )r r9 r? r` r r r r@ � s (z local.writec C sL t | jt�sH| j�|�sH| j�td�| � t� | j�
|�| j�
|�� dS )a� Link blobs found in the user cache into this store.
The server module needs to do this when it lets the client know not to
upload the blob, to ensure it is always available in this store.
Normally this is done implicitly when the client reads or writes the
blob, but that doesn't happen when the server tells the client that it
already has the blob.
� lfs: found %s in the usercache
N) �
isinstancerL r4 r
r: rD �noter r �linkr r8 r r r �linkfromusercache� s zlocal.linkfromusercachec C sL | j �|�sHt| j t�sH| j�td�| � t�| j �
|�| j �
|�� d S )Ns lfs: adding %s to the usercache
) rL r: rf r4 rD rg r r rh r
r r8 r r r r^ � s
�zlocal._linktousercacheTc C s� | j �|�sh| �| j||�}|s6tt�|��� �|kr�| j� t
d�| � t � | j�
|�| j �
|�� n$| j� t
d�| � | �| j ||�}|S )z Read blob from local blobstore.re s% lfs: found %s in the local lfs store
)r
r: �_readrL r rV rW r\ rD rg r r rh r )r r9 �verify�blobr r r r> � s z
local.readc C s |� |�}|rt||� |S )z0Read blob (after verifying) from the given store)r> �_verify)r r
r9 rk rl r r r rj � s
z local._readc C s` t �� }| �|��.}tj|dd�D ]}|�|� q"W d � n1 sF0 Y |t|�� �kS )zQIndicate whether or not the hash of the underlying file matches its
name.rS rT N)rV rW rP r rX rY r r\ )r r9 rW r` ra r r r rk � s
*z local.verifyc C s | j �|�p| j�|�S )zYReturns True if the local blobstore contains the requested blob,
False otherwise.)rL r: r
r8 r r r �has s z local.hasN)T)r0 r1 r2 rG r7 rP r rd r@ ri r^ r> rj rk rn r r r r rH p s
rH c C s� | }t | jt�r| j}t�|d�rjz|jjd }W n ttfyN |j}Y n0 t |tj �rft
� |�}|S t |dd�r�t
�
|j�S t�| �S dS )zbCreate a friendly message for the given URLError to be used in an
LfsRemoteError message.
s reason� �strerrorN)rf �reason� Exceptionr � safehasattr�args�AttributeError�
IndexErrorr
�unicoder Z
unitolocalr Z
strtolocalrp r � forcebytestr)�urlerror�instrq r r r �_urlerrorreason s
r{ c @ s e Zd ZdZdd� ZdS )�lfsauthhandleri� c C sR |� dd�}|rN|�� d }|�� dkrNtd�}tj�|�� |t� |�||��dS )z�Enforces that any authentication performed is HTTP Basic
Authentication. No authentication is also acceptable.
zwww-authenticateNr Zbasics, the server must support Basic Authentication)
�get�split�lowerr r �urlerr� httperrorZ get_full_urlr Z strfromlocal)r Zreqr` �coderb �headersZauthreq�schemer r r �http_error_401" s �z lfsauthhandler.http_error_401N)r0 r1 r2 Z
handler_orderr� r r r r r| s r| c @ sT e Zd Zdd� Zdd� Zdd� Zdd� Zd d
� Zd d � Zd
d� Z dd� Z
dd� Z dS )�
_gitlfsremotec C sr |j }|| _ |�� \}}|�d�| _|j �dd�}|sBdt�� }t�|||�| _ | j �
t � � |� dd�| _
d S )N� /rI s lfs.user-agents git-lfs/2.3.4 (Mercurial %s)rJ s retry)rD �authinfo�rstrip�baseurl�configr �version�urlmod�opener� urlopenerZ add_handlerr| Z configint�retry)r rM r rD r� r� Z useragentr r r r7 7 s z_gitlfsremote.__init__c C s | � t|�|d� dS )z,Batch upload from local to remote blobstore.� uploadN��_batch� _deduplicate�r �pointers� fromstorer r r �
writebatchC s z_gitlfsremote.writebatchc C s | � t|�|d� dS )z-Batch download from remote to local blostore.� downloadNr� �r r� �tostorer r r � readbatchG s z_gitlfsremote.readbatchc C sP dd� |D �}t �t�|t �|�d���}d| j }tjjt �|�|d�}|� dd� |� dd� z>t
� | j �
|���}|�� }W d � n1 s�0 Y W n� tjj�y } zbtd
�| j|f td �| j d �}
|
�| jtd
�||f �} ttd�t�| � | d��W Y d } ~ nXd } ~ 0 tjj�yn } z2td �| j } ttd�t| � | d��W Y d } ~ n
d } ~ 0 0 zt �|�} W n* t�y� ttd�|�d� ��Y n0 | j j �r<| j � d|j � t � |�!� ��"� }
| j � dd�#t$|
�%� �� � d| v �rt$| d dd� d�| d< | j � dt �tj| ddd d �� � d d � }t �&|| �S ) z�Get metadata about objects pointed by pointers for given action
Return decoded JSON object like {'objects': [{'oid': '', 'size': 1}]}
See https://github.com/git-lfs/git-lfs/blob/master/docs/api/batch.md
c S s$ g | ] }t �|�� �|�� d ��qS ))r9 rU )r
�strurlr9 rU �r! �pr r r r% Q s �z/_gitlfsremote._batchrequest.<locals>.<listcomp>)�objectsZ operations %s/objects/batch)r? ZAcceptz application/vnd.git-lfs+json� Content-TypeNs= check that lfs serving is enabled on %s and "%s" is supporteds/ the "lfs.url" config may be used to override %s)i� � s api=%s, action=%ss LFS HTTP error: %s��hint�
LFS error: %ss# LFS server returns invalid JSON: %s�utf-8� Status: %d
� %s
�
r� c S s | d S )Nr9 r )r� r r r �<lambda>� r z-_gitlfsremote._batchrequest.<locals>.<lambda>��keyr )� z: T)�indentZ
separatorsZ sort_keysc S s t | tj�r| �d�S | S )Nr� )rf r
rw �encode)�xr r r � encodestr� s
z._gitlfsremote._batchrequest.<locals>.encodestr)'r
�bytesurl�json�dumpsr� r� r �urlreq�request�
add_header�
contextlib�closingr� rP r> r� r� r r} r� r[ r rx ry r{ Z
json_loads�
ValueErrorr� rD � debugflag�debug�status�bytestr�info�stripr �sorted�
splitlinesZrapply)r r� �actionr� Z requestdatar ZbatchreqZrspZrawjson�ex�hintsr� �responser� r� r r r �
_batchrequestK s� ����
*���� ���� ��
�
���� z_gitlfsremote._batchrequestc C s� |D ]�}d|vr2|dkr||� dg �vrd}qDqn|� d�� dd�}dd� |D �}|� |d d
�}|r�t|d d �}d
dddd�} | � |d| �}
ttd�||
f ��qttd�|d ��qd
S )zUScans errors from objects
Raises LfsRemoteError if any objects have an errors errorr� � actionsr� s code�� c S s i | ]}|� � |�qS r )r9 r� r r r �
<dictcomp>� r z6_gitlfsremote._checkforservererror.<locals>.<dictcomp>� oidNrE s unknowns The object does not exists# The object was removed by the owners Validation errors Internal server error)r� i� i� r� s status code %ds LFS server error for "%s": %ss1 LFS server error. Unsolicited response for oid %s)r} r r[ r ) r r� Z responsesr� r� r� Zptrmapr� rE �errorsrb r r r �_checkforservererror� s4 � ����z"_gitlfsremote._checkforservererrorc s0 |� dg �}| �||� � � fdd�|D �}|S )z�extract objects from response of the batch API
response: parsed JSON object returned by batch API
return response['objects'] filtered by action
raise if any object has an error
s objectsc s g | ]}� |� d g �v r|�qS )r� �r} )r! �o�r� r r r% � s z1_gitlfsremote._extractobjects.<locals>.<listcomp>)r} r� )r r� r� r� r� Zfilteredobjectsr r� r �_extractobjects� s
�z _gitlfsremote._extractobjectsc
C s� |d }|d | � d�}|d | � di ��� }tj�t�|��}|dkrp|�|�sptj t
d�| t
d�d��|D ] \}} |� t�|�t�| �� qt�z�z8|dkr�t | j
|�|��|_d d
� |_|� d d � |� d
|jj� t�| j�|����}
|
�� � d�} | j
} | j
j�rP| �d|
j � t�|
�� ��� }| �dd� t |� � �� � |dk�rj|� ||
| � nHg }
|
� d�}|�s��q�|
�!|� �qnd� |
�}|�r�| �d||f � W d � n1 �s�0 Y W n� tj"j#�y< } zJ| j
j�r | j
�d||� � f � t$t
d�t%�&|�||f ��W Y d}~ndd}~0 tj"j'�y� } z>t
d�t�(tj)�*|�� }t$t
d�t+|� |d��W Y d}~n
d}~0 0 W |j�r�|j�,� n|j�r�|j�,� 0 dS )aC Download or upload a single object using basic transfer protocol
obj: dict, an object description returned by batch API
action: string, one of ['upload', 'download']
localstore: blobstore.local
See https://github.com/git-lfs/git-lfs/blob/master/docs/api/ basic-transfers.md
r� r� s hrefs headerr� � detected corrupt lfs object: %s�
run hg verifyr� c S s dS )NZPUTr r r r r r� � r z._gitlfsremote._basictransfer.<locals>.<lambda>r� zapplication/octet-streamzContent-Lengths content-lengthr� r� r� r� rS r s lfs %s response: %sNs %s: %s
s& LFS HTTP error: %s (oid=%s, action=%s)s attempted connection to %sr� )-r} �itemsr r� r� r
r� rk r �Abortr r� rA rD r r? Z
get_method�lengthr� r� r� rP r� r� r� r� r� r� r r� r� rd r> �appendr� r� r[ r rx ry r� Z urllibcompatZ
getfullurlr{ �close)r �objr� �
localstorer9 Zhrefr� r� �k�v�resZ
contentlengthrD Zblocksr? r� r� r� r r r �_basictransfer� sp
�
6
���
�� �z _gitlfsremote._basictransferc
s� � dvrt �d� ����|� �}��||� �}tdd� |D ��}i }|D ]}|�dd�||�d�< qJtd�td �d�� } t|�d
kr��j� td �t|�t
� |�f � � ��fd d
�}
�j� dd�r�t
�
�jd|
dt|dd� d��} n|
t|dd� d��} �jj| td�|d��d} | �d� d}
d}| D ]<\}}|
|| 7 }
|d
7 }| �|
� �j� td�| � �q,W d � n1 �s�0 Y |dk�r� dk�r��j�td�|t
� |
�f � n(� dk�r�j�td �|t
� |
�f � d S ) N)r� r� s invalid Git-LFS action: %sc s s | ]}|� d d�V qdS )� sizer Nr� )r! r� r r r � <genexpr>0 r z'_gitlfsremote._batch.<locals>.<genexpr>r� r r� s
lfs uploadings lfs downloadingro s& lfs: need to transfer %d objects (%s)
c 3 s� | D ]�}|� dd�}�jjr^� dkr.td�}n� dkr>td�}�j�||� d�t�|�f � �j}z&��|� �� d|� d�fV W qW qd t j
y� } zH|dkrֈj�td �t � |�|f � |d8 }W Y d }~qd� W Y d }~qdd }~0 0 qdqd S )
Nr� r r� s lfs: downloading %s (%s)
r� s lfs: uploading %s (%s)
r� ro s% lfs: failed: %r (remaining retry %d)
)
r} rD �verboser rg r � bytecountr� r� �socketr r rx )ra r� Zobjsizerb r� r� �r� r� r r r �transfer> s2
� �� z&_gitlfsremote._batch.<locals>.transferrI s lfs.worker-enableg�������?r c S s
| � d�S �Nr� r� �r� r r r r� ` r z&_gitlfsremote._batch.<locals>.<lambda>r� c S s
| � d�S r� r� r� r r r r� c r s bytes)Zunit�totals lfs: processed: %s
r� s lfs: uploaded %d files (%s)
r� s lfs: downloaded %d files (%s)
)r r r� r� �sumr} r r) rD rg r r� rK r r� Z makeprogressrY r� )r r� r� r� r� r� r� Zsizesr� Ztopicr� r- ZprogressZ processedZblobsZ_oner9 r r� r r� * sl �� �� �
�
8
��
��z_gitlfsremote._batchc C s: t | dd �}|r6|jD ] }|�� t |ddd� �� qd S )Nr� Z close_allc S s d S r5 r r r r r r� � r z'_gitlfsremote.__del__.<locals>.<lambda>)r �handlersr� )r r� �hr r r �__del__} s
z_gitlfsremote.__del__N) r0 r1 r2 r7 r� r� r� r� r� r� r� r� r r r r r� 6 s U)NSr� c @ s( e Zd ZdZdd� Zdd� Zdd� ZdS ) � _dummyremotez,Dummy store storing blobs to temp directory.c C s |j �d|j�}t|�| _ d S )NrJ )r
r r r )r rM r rN r r r r7 � s z_dummyremote.__init__c C sb t |�D ]T}|j|�� dd�}| j|�� ddd��}|�|� W d � q1 sR0 Y qd S )NT)rk rQ rR )r� r> r9 r
r@ )r r� r� r� �contentr` r r r r� � s z_dummyremote.writebatchc C sT t |�D ]F}| �|�� d��"}|�|�� |d � W d � q1 sD0 Y qd S rB )r� r
r9 rd )r r� r� r� r` r r r r� � s z_dummyremote.readbatchN�r0 r1 r2 rG r7 r� r� r r r r r� � s r� c @ s( e Zd ZdZdd� Zdd� Zdd� ZdS ) � _nullremotez&Null store storing blobs to /dev/null.c C s d S r5 r �r rM r r r r r7 � s z_nullremote.__init__c C s d S r5 r r� r r r r� � s z_nullremote.writebatchc C s d S r5 r r� r r r r� � s z_nullremote.readbatchNr� r r r r r� � s r� c @ s4 e Zd ZdZdd� Zd dd�Zd dd�Zd d
� ZdS )
�
_promptremotez)Prompt user to set lfs.url when accessed.c C s d S r5 r r� r r r r7 � s z_promptremote.__init__Nc C s | � � d S r5 ��_prompt)r r� r� rD r r r r� � s z_promptremote.writebatchc C s | � � d S r5 r� )r r� r� rD r r r r� � s z_promptremote.readbatchc C s t �td���d S )Ns lfs.url needs to be configured)r r� r r6 r r r r� � s z_promptremote._prompt)N)N)r0 r1 r2 rG r7 r� r� r� r r r r r� � s
r� )� https� https files nullNc C s& t �� }| D ]}|||�� < q |�� S )z0Remove any duplicate oids that exist in the list)r Zsortdictr9 �values)r� Zreducedr� r r r r� � s r� c C s6 t t�|��� �}|| kr2ttd�| td�d��d S )Nr� r� r� )r rV rW r\ r] r )r9 r� rc r r r rm � s
�rm c C s� | j �dd�}t�|pd�}|du r�|r.|}n&t�| d�rB| j}n| j �dd�pRd}t�|�}|jdv r�|jr�|jdd � d
kr�| jd
7 _|jp�dd |_t�t |��}| j �
t d �| � |j}|t vr�t
�t d
�| ��t | | |�S )a� remotestore factory. return a store in _storemap depending on config
If ``lfs.url`` is specified, use that remote endpoint. Otherwise, try to
infer the endpoint, based on the remote repository using the same path
adjustments as git. As an extension, 'http' is supported as well so that
``hg serve`` works out of the box.
https://github.com/git-lfs/git-lfs/blob/master/docs/api/server-discovery.md
rJ s urlr Ns _subtoppaths pathss default)r� r� ���r� s
.git/info/lfss lfs: assuming remote store: %s
s lfs: unknown url scheme: %s)rD r� r r r rs Z _subtoppathr� r �bytesrg r � _storemapr r� )rM �remoteZlfsurlr r Z
defaulturlr� r r r r� � s&
r� c @ s e Zd ZdS )r[ N)r0 r1 r2 r r r r r[ � s r[ c @ s e Zd ZdZdS )r] z~Raised when a corrupt blob is detected, aborting an operation
It exists to allow specialized handling on the server side.N)r0 r1 r2 rG r r r r r] � s r] )N)9Z
__future__r r� r< rV r� r& �rer� Zmercurial.i18nr Zmercurial.pycompatr Zmercurial.noder Z mercurialr r r Zhttpconnectionmodr r
r r� r r
Zvfsmodr Zmercurial.utilsr r Z
largefilesr �compiler r r4 Z httpsendfilerA �objectrH r{ r� Z basehandlerr| r� r� r� r� r� r� rm r� Z StorageErrorr[ r� r] r r r r �<module> sL ,
R
�
+