HOME

�
y]Rc@s�
dd
lZdd
l
Z
dd
lZdd
lZdd
lZd
ZyddlmZ
Wn#e	k
r{
Z

ddlmZ
n
Xdef
d��YZdej
f
d��YZdef
d��YZd	dd
��YZdejf
d��YZdd
lmZmZmZmZmZmZmZ
dd
lZdd
lZdd
lZdd
lZdd
lZdejf
d��YZ dej!f
d��YZ!d
a"d�Z#d�Z$d�Z%d
S(i����N(
tStringIOt
RangeErrorc
BseZ
dZRS(
s6Error raised when an unsatisfiable range is requested.(t__name__t
__module__t__doc__(((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR
"s

tHTTPRangeHandlerc
Bs eZ
dZd
�Zd�ZRS(s�Handler that enables HTTP Range headers.
    
    This was extremely simple. The Range header is a HTTP feature to
    begin with so all this class does is tell urllib2 that the 
    "206 Partial Content" response from the HTTP server is what we 
    expected.
    
    Example:
        import urllib2
        import byterange
        
        range_handler = range.HTTPRangeHandler()
        opener = urllib2.build_opener(range_handler)
        
        # install it
        urllib2.install_opener(opener)
        
        # create Request and set Range header
        req = urllib2.Request('http://www.python.org/')
        req.header['Range'] = 'bytes=30-50'
        f = urllib2.urlopen(req)
    cCs1tj
|||
j��}||_||_|S(
N(turllibt
addinfourltget_full_urltcodetmsg(tselftreqtfpR	R
thdrst
r((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pythttp_error_206>s
	
	
cCstd
d��
dS(Ni	sRequested Range Not Satisfiable(
R
(RRR
R	R
R((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pythttp_error_416Es(RRRRR(((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR&s	tHTTPSRangeHandlerc
Bs eZ
dZd
�Zd�ZRS(s! Range Header support for HTTPS. cCs|j|
||||�S(
N(
R(RRR
R	R
R((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pythttps_error_206Ls
cCs|j|
||||�
dS(
N(
thttps_error_416(RRR
R	R
R((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyROs
(RRRRR(((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyRIs
	tRangeableFileObjectcBsheZ
dZd
�Zd�Zd�Zdd�
Zdd�
Zdd�
Zd	�Z	d
�Z
d�ZRS(s"File object wrapper to enable raw range handling.
    This was implemented primarilary for handling range 
    specifications for file:// urls. This object effectively makes 
    a file object look like it consists only of a range of bytes in 
    the stream.
    
    Examples:
        # expose 10 bytes, starting at byte position 20, from 
        # /etc/aliases.
        >>> fo = RangeableFileObject(file('/etc/passwd', 'r'), (20,30))
        # seek seeks within the range (to position 23 in this case)
        >>> fo.seek(3)
        # tell tells where your at _within the range_ (position 3 in
        # this case)
        >>> fo.tell()
        # read EOFs if an attempt is made to read past the last
        # byte in the range. the following will return only 7 bytes.
        >>> fo.read(30)
    cCs>|
|_t
|�
\|_|_d
|_|j|j�

dS(s
Create a RangeableFileObject.
        fo       -- a file like object. only the read() method need be 
                    supported but supporting an optimized seek() is 
                    preferable.
        rangetup -- a (firstbyte,lastbyte) tuple specifying the range
                    to work over.
        The file object provided is assumed to be at byte offset 0.
        iN(tfotrange_tuple_normalizet	firstbytetlastbytetrealpost_do_seek(RRtrangetup((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyt__init__gs		

	
cCs/t|j
|
�r"t|j
|
�St|
�d
S(s�This effectively allows us to wrap at the instance level.
        Any attribute not found in _this_ object will be searched for
        in self.fo.  This includes methods.N(thasattrRtgetattrtAttributeError(Rtname((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyt__getattr__us

c

Cs|j|j
S(
s5
Return the position within the range.
        This is different from fo.seek in that position 0 is the 
        first byte position of the range tuple. For example, if
        this object was created with a range tuple of (500,899),
        tell() will return 0 when at byte position 500 of the file.
        (RR(
R((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyttell}sicCs�|dkst�
|d
kr.|j
|
}n7|dkrJ|j|
}n|dkretd�
�
n|jr�||jkr�|j}n|j||j�

dS(seSeek within the byte range.
        Positioning is identical to that described under tell().
        ii
is$seek from end of file not supported.N(ii
i(tAssertionErrorRRtIOErrorRR(Rtoffsettwhencet
realoffset((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pytseek�s





i����cCs:|j|
�
}
|j
j|
�
}|jt|�
7_|S(
s`Read within the range.
        This method will limit the size read based on the range.
        (t_calc_read_sizeRtreadRtlen(Rtsizetrslt((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR+�s


cCs:|j|
�
}
|j
j|
�
}|jt|�
7_|S(
sfRead lines within the range.
        This method will limit the size read based on the range.
        (R*RtreadlineRR,(RR-R.((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR/�s


cCsX|jrT|
d
krA|j
|
|jkrQ|j|j
}
qQqT|j|j
}
n|
S(sSHandles calculating the amount of data to read based on
        the range.
        i����(RR(RR-((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR*�s	



cCs^|
d
kst�
t
|jd�s4|j|
�

n|jj|j|
�

|j|
7_dS(s�Seek based on whether wrapped object supports seek().
        offset is relative to the current position (self.realpos).
        iR)N(R$RRt_poor_mans_seekR)R(RR&((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR�s



cCs�d
}d}xm||
kr{|||
kr8|
|}n|jj
|�
}t|�
|krntdd��
n||7}qWdS(s�
Seek by calling the wrapped file objects read() method.
        This is used for file like objects that do not have native
        seek support. The wrapped objects read() method is called
        to manually seek to the desired position.
        offset -- read this number of bytes from the wrapped
                  file object.
        raise RangeError if we encounter EOF before reaching the 
        specified offset.
        iii	sRequested Range Not SatisfiableN(RR+R,R
(RR&tpostbufsizetbuf((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR0�s









(RRRRR"R#R)R+R/R*RR0(((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyRRs								tFileRangeHandlerc
BseZ
dZd
�ZRS(s~FileHandler subclass that adds Range support.
    This class handles Range headers exactly like an HTTP
    server would.
    c
Cs�
d
dl}d
dl
}|
j�}|
j�}tj|�
}tj|�
}|tj}t	j
|tj�
}	|j|�
d}
|r�tj
|�
\}}|s�tj|�
|j�kr�tjd�
�
q�nt|d�}|
jjdd�}
t|
�
}
|
dks
t�
|
r�
|
\}}|dkr>
|}n|dksb
||ksb
||krt
tdd��
n||}t|||f�}n|jtd	|
p�
d
||	f�
�
}tj||d|�S(
Ni����isfile not on local hosttrbtRangeti	sRequested Range Not Satisfiables6Content-Type: %s
Content-Length: %d
Last-modified: %s
s
text/plainsfile:((t	mimetypest	mimetoolstget_hosttget_selectorRturl2pathnametoststattST_SIZEtrfc822t
formatdatetST_MTIMEt
guess_typet	splitporttsockett
gethostbynamet	get_namesturllib2tURLErrortopentheaderstgettNonetrange_header_to_tupleR$R
RtMessageRR(RRR8R9thosttfilet	localfiletstatsR-tmodifiedtmtypetportRtbrangetfbtlbRK((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pytopen_local_file�s:












!







	
$




	


(RRRRZ(((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR4�s
(RDt	splitusertsplitpasswdt	splitattrtunquotetaddclosehookRtFTPRangeHandlerc
BseZ
d�Zd
�ZRS(cCs�|
j�}|st
d�nt|�
\}}|dkrHtj}nt|�
}t|�
\}}|r�t|�
\}}nd}t	|�
}t	|p�d�
}t	|p�d�
}yt
j|�
}Wn%t
jk
r�
}
t
j|�
�
n
Xt|
j��
\}}|jd�
}	tt	|	�}	|	d |	d}	}
|	r[
|	dr[
|	d}	ny$|j|||||	�}|
r�
dp�
d	}xM|D]E}
t|
�
\}
}|
j�d
kr�
|dkr�
|j�}q�
q�
Wd}t|
jjdd��
}|dkst�
|r8|\}}|dkr8|}q8n|j|
||�\}}|r�|\}}|dkr�|dks�|dkr�tdd��
n|}||}|dkr�tdd��
q�q�||}t|d|f�}nd}tj|
j��
d}|r$|d|7}n|dk	rM|dkrM|d|7}nt |�
}t!j"|�
}t#|||
j��SWn2tj$k
r�
}
t
d
|ft%j&�d�n
XdS(Ns	ftp errors
no host givenR7t
/i����ii
t
It
Dttypet
at
At
it
dR6i	s@Requested Range Not Satisfiable due to unobtainable file length.sRequested Range Not SatisfiablesContent-Type: %s
sContent-Length: %d
i(s	ftp errors
no host given(ReRfRgRbRhRc(('R:R%RDRMtftplibtFTP_PORTtintR[R\R^RERFterrorRHRIR]R;tsplittmaptconnect_ftptlowertupperRNRKRLR$tretrfileR
RR8RCRRR9RORt
all_errorstsystexc_info(RRRPRVtusertpasswdR
tpathtattrstdirsRQtfwRdtattrtvaluetrestt	range_tupRXRYR
tretrlenRKRUtsf((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pytftp_open

s|






















































cCst|
||||�}|S(
N(
t
ftpwrapper(RRvRwRPRVRzR{((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyRoX
s

(RRR�Ro(((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR`	
s
	NR�cBseZ
d
d�
ZRS(c
Cs|j�
|dkr%d}d}nd|}d}y|j
j|�

Wn.tjk
ry


|j�
|j
j|�

n
Xd}|
r�
|r�
y|j
j|
�

Wn2tjk
r�
}
t	d|ft
j�d�n
X|j
j|�

y#d	|
}|j
j||�}Wq�
tjk
r�

}
t
|�
d
 dkrk
|j|
|�\}}	t||df�}||	fSt
|�
d
 d
kr�
t	d|ft
j�d�q�
q�
Xn|s�
|j
jd�

|
r�
d|
}nd}|j
j|�
}nd|_t|djd�
|j�|dfS(NRhRcsTYPE Ai
sTYPE is	ftp errorisRETR it501R7t550sLIST tLISTR5(RhRc(tendtransfertftptvoidcmdRiRstinitRMtnlstt
error_permR%RtRutntransfercmdtstrRrRtbusyR_tmakefile(
RRQRdR~tcmdtisdirtconntreasonR
R�((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyRra
sJ


	













 









&




	
N(RRRMRr(((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR�\
sc
Cs�|dkrdSt
dkr:d
dl}
|
jd�
a
nt
j|�
}|r�t|jdd��
}|r�|dr�|d|ddf}n|SdS(s�
Get a (firstbyte,lastbyte) tuple from a Range header value.
    
    Range headers have the form "bytes=<firstbyte>-<lastbyte>". This
    function pulls the firstbyte and lastbyte values and returns
    a (firstbyte,lastbyte) tuple. If lastbyte is not specified in
    the header value, it is returned as an empty string in the
    tuple.
    
    Return None if range_header is None
    Return () if range_header does not conform to the range spec 
    pattern.
    
    i����Ns^bytes=(\d{1,})-(\d*)i
ii((RMt_rangeretretcompiletmatchRtgroup(trange_headerR�R�ttup((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyRN�
s









c

CsS|dkrdSt
|�
}|rO|d
rG|d|d
d
f}nd|SdS(s�Convert a range tuple to a Range header value.
    Return a string of the form "bytes=<firstbyte>-<lastbyte>" or None
    if no range is needed.
    i
isbytes=%s-%sN(RMR(
R((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pytrange_tuple_to_header�
s





c
Cs�|dkrdS|d
}
|
dkr/d
}
nt
|
�
}
y|d}Wntk
rb


d}n1X|dkrxd}n|dkr�t
|�
}n|
|fdkr�dS||
kr�tdd|
|f��
n|
|fS(	s7
Normalize a (first_byte,last_byte) range tuple.
    Return a tuple whose first element is guaranteed to be an int
    and whose second element will be '' (meaning: the last byte) or 
    an int. Finally, return None if the normalized tuple == (0,'')
    as that is equivalent to retrieving the entire file.
    iR7i
i	sInvalid byte range: %s-%sN(NR7(iR7(RMRkt
IndexErrorR
(RRXRY((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyR�
s&

	



	

((&R=R>RRHR@RMtDEBUGt	cStringIORtImportErrorR
R%R
tBaseHandlerRRRtFileHandlerR4RDR[R\R]R^R_RRiRERtR8R9t
FTPHandlerR`R�R�RNR�R(((s8/usr/lib/python2.7/site-packages/urlgrabber/byterange.pyt<module>s4






#	�+4



S4