HOME

Mini Shell 1.0
DIR:/opt/alt/python38/lib64/python3.8/site-packages/playhouse/__pycache__/
Current File : //opt/alt/python38/lib64/python3.8/site-packages/playhouse/__pycache__/csv_utils.cpython-38.pyc
U

S��WX-�@s�dZddlZddlZddlZddlZddlmZzddlmZWn ek
r`ddl	mZYnXddl
Tddl
mZddl
mZddl
m
Z
e
r�eZd	Znd
ZGdd�de�Zd
d�ZGdd�de�ZGdd�de�Zddd�Zeje_ddd�ZdS)a
Peewee helper for loading CSV data into a database.

Load the users CSV file into the database and return a Model for accessing
the data:

    from playhouse.csv_loader import load_csv
    db = SqliteDatabase(':memory:')
    User = load_csv(db, 'users.csv')

Provide explicit field types and/or field names:

    fields = [IntegerField(), IntegerField(), DateTimeField(), DecimalField()]
    field_names = ['from_acct', 'to_acct', 'timestamp', 'amount']
    Payments = load_csv(db, 'payments.csv', fields, field_names)
�N)�contextmanager)�StringIO)�*)�Database)�Func)�PY3FTc@seZdZedd��ZdS)�
_CSVReadercksjd}t|t�rt|d�}n,t|t�r4|}|�d�n|}|�d�d}tj|f|�}|V|rf|��dS)NF�rrT)�
isinstance�
basestring�openr�seek�csv�reader�close)�self�file_or_name�
reader_kwargs�is_file�fhr�r�
/csv_utils.py�
get_reader's


z_CSVReader.get_readerN)�__name__�
__module__�__qualname__rrrrrrr&srcs��fdd�}|S)Ncs��fdd�|_|S)Ncs
�f��S�Nrr��field_class�field_kwargsrr�<lambda>:�z2convert_field.<locals>.decorator.<locals>.<lambda>)�field)�fnrrr�	decorator9sz convert_field.<locals>.decoratorr)rrr$rrr�
convert_field8sr%c@s�eZdZdZddgZddgZd!dd	�Zd
d�Zee	dd
�dd��Z
eedd
�dd��Zee
dd�dd��Zeedd�dd��Zeedd
�dd��Zdd�Zdd�Zdd�Zd S)"�RowConvertera
    Simple introspection utility to convert a CSV file into a list of headers
    and column types.

    :param database: a peewee Database object.
    :param bool has_header: whether the first row of CSV is a header row.
    :param int sample_size: number of rows to introspect
    z%Y-%m-%dz%m/%d/%Yz%Y-%m-%d %H:%M:%Sz%Y-%m-%d %H:%M:%S.%fT�
cCs||_||_||_dSr)�database�
has_header�sample_size)rr(r)r*rrr�__init__OszRowConverter.__init__c	Cs<|D]2}ztj�||�Wntk
r.YqXdSqdS�NT)�datetime�strptime�
ValueError)r�valueZformatsZfmtrrr�matches_dateTszRowConverter.matches_dater)�defaultcCs|��Sr)�isdigit�rr0rrr�
is_integer]szRowConverter.is_integerc	Cs.zt|�Wnttfk
r$YnXdSdSr,)�floatr/�	TypeErrorr4rrr�is_floatas
zRowConverter.is_float)ZnullcCs|�||j�Sr)r1�datetime_formatsr4rrr�is_datetimejszRowConverter.is_datetimecCs|�||j�Sr)r1�date_formatsr4rrr�is_datenszRowConverter.is_date�cCsdSr,rr4rrrr2rszRowConverter.defaultc		Ks�g}|j}|j|f|��>}|jr(|d7}t|�D] \}}|�|�||jkr0qRq0W5QRX|jrz|d|dd�}}ndd�tt|d��D�}||fS)aq
        Extract `self.sample_size` rows from the CSV file and analyze their
        data-types.

        :param str file_or_name: A string filename or a file handle.
        :param reader_kwargs: Arbitrary parameters to pass to the CSV reader.
        :returns: A 2-tuple containing a list of headers and list of rows
                  read from the CSV file.
        �rNcSsg|]}d|�qS�zfield_%dr��.0�irrr�
<listcomp>�sz-RowConverter.extract_rows.<locals>.<listcomp>)r*rr)�	enumerate�append�range�len)	rrr�rowsZrows_to_readrrB�row�headerrrr�extract_rowsvs


zRowConverter.extract_rowscCs|j|j|j|j|jgS)z6Return a list of functions to use when testing values.)r<r:r5r8r2�rrrr�
get_checks�s�zRowConverter.get_checksc	srt|�}|��}g}t|�D]P\}}dd�|D�}|D]4�t�fdd�|D��}t|�r6|�����qq6q|S)a
        Analyze the given rows and try to determine the type of value stored.

        :param list rows: A list-of-lists containing one or more rows from a
                          csv file.
        :returns: A list of peewee Field objects for each column in the CSV.
        cSsg|]}|dkr|�qS)r=r�rA�valrrrrC�sz(RowConverter.analyze.<locals>.<listcomp>c3s|]}�|�VqdSrrrN�Zcheckrr�	<genexpr>�sz'RowConverter.analyze.<locals>.<genexpr>)�ziprMrD�set�allrEr")	rrHZ
transposedZchecksZcolumn_typesrB�columnZcol_vals�resultsrrPr�analyze�szRowConverter.analyzeN)Tr')rrr�__doc__r;r9r+r1r%�IntegerFieldr5Z
FloatFieldr8Z
DateTimeFieldr:Z	DateFieldr<Z	BareFieldr2rKrMrWrrrrr&>s,	��
	









	r&c@sBeZdZdZddd�Zdd	�Zd
d�Zdd
�Zdd�Zdd�Z	dS)�Loaderat
    Load the contents of a CSV file into a database and return a model class
    suitable for working with the CSV data.

    :param db_or_model: a peewee Database instance or a Model class.
    :param file_or_name: the filename of the CSV file *or* a file handle.
    :param list fields: A list of peewee Field() instances appropriate to
        the values in the CSV file.
    :param list field_names: A list of names to use for the fields.
    :param bool has_header: Whether the first row of the CSV file is a header.
    :param int sample_size: Number of rows to introspect if fields are not
        defined.
    :param converter: A RowConverter instance to use.
    :param str db_table: Name of table to store data in (if not specified, the
        table name will be derived from the CSV filename).
    :param reader_kwargs: Arbitrary arguments to pass to the CSV reader.
    NTr'Fc
Ks�||_||_||_||_||_||_|
|_t|t�r<||_	nt|t
�rNd|_	n|j|_	t|t�r�||_
d|_|p�tj�tj�|j	��d|_nd||_|jjj
|_
|jjj|_|jjj|_|jjj|_|jjjr�|	s�|jdd�|_|jdd�|_dS)Nzdata.csvrr>)r�fields�field_namesr)r*�	converterrr
r�filenamer�namerr(�model�os�path�splitext�basename�db_table�_metaZ
sorted_fieldsZsorted_field_namesZauto_increment)r�db_or_modelrr[r\r)r*r]re�	pk_in_csvrrrrr+�s4


�zLoader.__init__cCst�dd|���S)Nz
[^a-z0-9]+�_)�re�sub�lower)r�srrr�clean_field_name�szLoader.clean_field_namecCs|jpt|j|j|jd�S)N)r)r*)r]r&r(r)r*rLrrr�
get_converter�s
�zLoader.get_convertercsb�����j�jf�j�\}}|r0��|��_n�fdd�|D��_�js^�fdd�|D��_dS)Ncsg|]}�j���qSr)r2r")rAri)r]rrrC�sz&Loader.analyze_csv.<locals>.<listcomp>csg|]}��|��qSr�rn�rA�colrLrrrC�s)rorKrrrWr[r\)rrJrHr)r]rr�analyze_csv�s��zLoader.analyze_csvcCsr|jr|jStt||��}d|kr.t�|d<nt|dt�rFt�|d<t|j��t	f|�}|j
|j_
|j|j_|S)N�idZ_auto_pk)r`�dictrRZPrimaryKeyFieldr
rY�typere�titleZModelr(rf)rr\r[Zattrs�klassrrr�get_model_class�s


zLoader.get_model_classc
s�js����js4�js4dd�tt�j��D��_�j�jf�j�}|��}�jspt	|�}�fdd�|D��_n�jr~t	|���
�j�j�}�j���b|�
d�|D]L}i}t�j|�D]"\}}|r�tr�|�d�}|||<q�|r�|jf|���q�W5QRXW5QRX|S)NcSsg|]}d|�qSr?rr@rrrrCszLoader.load.<locals>.<listcomp>csg|]}��|��qSrrprqrLrrrCsTzutf-8)r[rsr\r)rFrGrrr�nextryr(ZtransactionZcreate_tablerR�decode_value�decode�insertZexecute)rZ
reader_objrrIZ
ModelClassr}Z
field_namer0rrLr�loads4�


&zLoader.load)NNTr'NNF)
rrrrXr+rnrorsryr~rrrrrZ�s�
%
rZr'c	Ks*tf|||||||||d�	|	��}
|
��S)N)	rgrr[r\r)r*r]rerh)rZr~)rgrr[r\r)r*r]rerhr�loaderrrr�load_csv's�
�r�cCs�t|t�rt||rdpd�}n|}|r2|�dd�|pHtj|ddtjd�}|r�g}t|j�D]F\}	}
|
j	rx|�
|
j	�q\t|
ttf�r�|�
|
j
�q\|�
d|	�q\|�|�|����D]}|�|�q�|r�|��|S)	z'
    Create a CSV dump of a query.
    �a�wr��,�")Z	delimiterZ	quotecharZquotingzcol_%s)r
rrr
r�writerZ
QUOTE_MINIMALrDZ_selectZ_aliasrEZFieldrr_ZwriterowZtuples�iteratorr)ZqueryrZinclude_headerZ
close_filerEZ
csv_writerrr�rJ�idxZnoderIrrr�dump_csv9s2
�
r�)NNTr'NNF)TTTN)rXrr-rarj�
contextlibrr�ImportError�ioZpeeweerrr�strrr{�objectrr%r&rZr�r�rrrr�<module>sBqx�
�