Lc @ s d Z d d k l Z d Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k Z d d k
l Z d d k l
Z
l Z l Z d d k l Z l Z e d Z d
e f d YZ d Z d
Z d Z d Z e i d d f j o+ d d k Z e i Z d Z d Z n e Z e Z e Z d Z! d e f d YZ" d e# f d YZ$ d e f d YZ% d e$ f d YZ& d S( s Refactoring framework.
Used as a main program, this can refactor any number of files and/or
recursively descend down directories. Imported as a module, this
provides infrastructure to write your own refactoring tool.
i( t with_statements# Guido van Rossum N( t chaini ( t drivert tokenizet token( t pytreet pygramc C s t | g g d g } t i i | i } g } xg t t i | D]P } | i d o: | i d o* | o | d } n | i | d qI qI W| S( sE Return a sorted list of all available fix names in the given package.t *t fix_s .pyi i(
t
__import__t ost patht dirnamet __file__t sortedt listdirt
startswitht endswitht append( t fixer_pkgt
remove_prefixt pkgt fixer_dirt fix_namest name( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt get_all_fix_names s t
_EveryNodec B s e Z RS( ( t __name__t
__module__( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR + s c C s t | t i t i f o+ | i d j o
t n t | i g St | t i o" | i o t
| i St n t | t i oF t } x5 | i D]* } x! | D] } | i t
| q Wq W| St
d | d S( sf Accepts a pytree Pattern Node and returns a set
of the pattern types which will match first. s$ Oh no! I don't understand pattern %sN( t
isinstanceR t NodePatternt LeafPatternt typet NoneR t sett NegatedPatternt contentt _get_head_typest WildcardPatternt updatet Exception( t patt rt pt x( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR% / s"
c C s t i t } g } x | D] } | i o\ y t | i } Wn t j
o | i | q XxX | D] } | | i | qi Wq | i d j o | | i i | q | i | q Wx: t t
i i i
t
i i D] } | | i | q Wt | S( s^ Accepts a list of fixers and returns a dictionary
of head node type --> fixer list. N( t collectionst defaultdictt listt patternR% R R t _accept_typeR! R R t python_grammart
symbol2numbert
itervaluest tokenst extendt dict( t
fixer_listt
head_nodest everyt fixert headst node_type( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _get_headnode_dictK s(
c C s0 g } t | t D] } | | d | q ~ S( sN
Return the fully qualified names for fixers in the package pkg_name.
t .( R t False( t pkg_namet _[1]t fix_name( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt get_fixers_from_packaged s c C s | S( N( ( t obj( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _identityk s i i c C s | i d d S( Nu
u
( t replace( t input( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _from_system_newlinesr s c C s, t i d j o | i d t i S| Sd S( Ns
u
( R
t linesepRG ( RH ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _to_system_newlinest s c s t } t i t i | i f d } t t i t i t i f } t
} yx}t ou| \ } } | | j o q] q] | t i j o | o Pn t } q] | t i
j o| d j o
| \ } } | t i
j p
| d j o Pn | \ } } | t i
j p
| d j o Pn | \ } } | t i j o | d j o | \ } } n xg | t i
j oQ | i | | \ } } | t i j p
| d j o Pn | \ } } qnWq] Pq] WWn t j
o n Xt | S( Nc s i } | d | d f S( Ni i ( t next( t tok( t gen( s( /usr/lib64/python2.6/lib2to3/refactor.pyt advance s u fromu
__future__u importu (u ,( R@ R t generate_tokenst StringIOt readlinet frozensetR t NEWLINEt NLt COMMENTR" t Truet STRINGt NAMEt OPt addt
StopIteration( t sourcet have_docstringRO t ignoret featurest tpt value( ( RN s( /usr/lib64/python2.6/lib2to3/refactor.pyt _detect_future_features sH
t
FixerErrorc B s e Z d Z RS( s A fixer could not be loaded.( R R t __doc__( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyRd s t RefactoringToolc B s e Z h e d 6Z d Z d Z d d d Z d Z d Z d Z
d Z d Z e e d Z
e e d
Z d Z e e d Z d
Z e d Z d Z d Z d e d d Z d d Z d Z d Z d Z d Z d Z d Z d Z d Z RS( t print_functiont FixR c C s | | _ | p g | _ | i i | _ | d j o | i i | n | i d o t i | _ n
t i
| _ g | _ t i
d | _ g | _ t | _ t i | i d t i d | i | _ | i \ | _ | _ t | i | _ t | i | _ g | _ d S( s Initializer.
Args:
fixer_names: a list of fixers to import
options: an dict with configuration.
explicit: a list of fixers to run even if they are explicit.
Rg Rf t convertt loggerN( t fixerst explicitt _default_optionst copyt optionsR! R' R t! python_grammar_no_print_statementt grammarR2 t errorst loggingt getLoggerRj t fixer_logR@ t wroteR t DriverR Ri t
get_fixerst pre_ordert
post_orderR> t pre_order_headst post_order_headst files( t selft fixer_namesRo Rl ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt __init__ s&
c
C s g } g } x| i D]} t | h h d g } | i d d d } | i | i o | t | i } n | i d } | i d i g } | D] } | | i q ~ } y t
| | }
Wn) t j
o t d | | f n X|
| i
| i } | i o7 | i t j o' | | i j o | i d | q n | i d | | i d
j o | i | q | i d j o | i | q t d | i q Wt i d
} | i d | | i d | | | f S( s Inspects the options to load the requested patterns and handlers.
Returns:
(pre_order, post_order), where pre_order is the list of fixers that
want a pre-order AST traversal, and post_order is the list that want
post-order traversal.
R R? i it _t s Can't find %s.%ss Skipping implicit fixer: %ss Adding transformation: %st pret posts Illegal fixer order: %rt run_ordert key( Rk R t rsplitR t FILE_PREFIXt lent splitt CLASS_PREFIXt joint titlet getattrt AttributeErrorRd Ro Ru Rl RW t log_messaget log_debugt orderR t operatort
attrgettert sort(
R~ t pre_order_fixerst post_order_fixerst fix_mod_patht modRC t partsRB R+ t
class_namet fix_classR; t key_func( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyRx s:
7c O s d S( s Called when an error occurs.N( ( R~ t msgt argst kwds( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt log_error s c G s) | o | | } n | i i | d S( s Hook to log a message.N( Rj t info( R~ R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s c G s) | o | | } n | i i | d S( N( Rj t debug( R~ R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s c C s d S( sT Called with the old version, new version, and filename of a
refactored file.N( ( R~ t old_textt new_textt filenamet equal( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt print_output s c C sR xK | D]C } t i i | o | i | | | q | i | | | q Wd S( s) Refactor a list of files and directories.N( R
R t isdirt refactor_dirt
refactor_file( R~ t itemst writet
doctests_onlyt dir_or_file( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt refactor
s
c C s x t i | D] \ } } } | i d | | i | i xk | D]c } | i d oL t i i | d i d o, t i i | | } | i | | | qJ qJ Wg } | D]! }
|
i d p | |
q q ~ | (q Wd S( s Descends down a directory and refactor every Python file found.
Python files are assumed to have a .py extension.
Files and subdirectories starting with '.' are skipped.
s Descending into %sR? i t pyN(
R
t walkR R R R t splitextR R R ( R~ t dir_nameR R t dirpatht dirnamest filenamesR t fullnameRB t dn( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c C s y t | d } Wn) t j
o } | i d | | d SXz t i | i d } Wd | i Xt | d d | i i
} z# | ~ } t | i | f SWd QXd S( sG
Do our best to decode a Python source file correctly.
t rbs Can't open %s: %si NR* t encoding( NN(
t opent IOErrorR R! R t detect_encodingRR t closet _open_with_encodingt __exit__t __enter__RI t read( R~ R t ft errR RB ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt _read_python_source( s ,c C s | i | \ } } | d j o d S| d 7} | o` | i d | | i | | } | | j o | i | | | | | q | i d | n^ | i | | } | o4 | i o* | i t | d | d | d | n | i d | d S( s Refactors a file.Nu
s Refactoring doctests in %ss No doctest changes in %siR R s No changes in %s( R R! R t refactor_docstringt processed_filet refactor_stringt was_changedt unicode( R~ R R R RH R t outputt tree( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR 8 s
c C s t | } d | j o t i | i _ n zO y | i i | } Wn2 t j
o& } | i d | | i i | d SXWd | i | i _ X| | _
| i d | | i | | | S( sF Refactor a given input string.
Args:
data: a string holding the code to be refactored.
name: a human-readable name for use in error/log messages.
Returns:
An AST corresponding to the refactored input stream; None if
there were errors during the parse.
Rg s Can't parse %s: %s: %sNs Refactoring %s(
Rc R Rp R Rq t parse_stringR( R t __class__R t future_featuresR t
refactor_tree( R~ t dataR R` R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR O s
c C s t i i } | oT | i d | i | d } | | j o | i | d | q | i d nN | i | d } | o' | i o | i t | d | n | i d d S( Ns Refactoring doctests in stdins s No doctest changes in stdins No changes in stdin( t syst stdinR R R R R R R ( R~ R RH R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt refactor_stdinj s
c C s x- t | i | i D] } | i | | q W| i | i | i | i | i | i x- t | i | i D] } | i | | qx W| i S( sA Refactors a parse tree (modifying the tree in place).
Args:
tree: a pytree.Node instance representing the root of the tree
to be refactored.
name: a human-readable name for this tree.
Returns:
True if the tree was modified, False otherwise.
( R Ry Rz t
start_treet traverse_byR{ R| t finish_treeR ( R~ R R R; ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR z s c C s | p d Sxv | D]n } xe | | i D]V } | i | } | o: | i | | } | d j o | i | | } q} q' q' Wq Wd S( s Traverse an AST, applying a set of fixers to each node.
This is a helper method for refactor_tree().
Args:
fixers: a list of fixer instances.
traversal: a generator that yields AST nodes.
Returns:
None
N( R t matcht transformR! RG ( R~ Rk t traversalt nodeR; t resultst new( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c C s | i i | | d j o) | i | d } | d j o d Sn | | j } | i | | | | | o | i d | d S| o | i | | | | n | i d | d S( sP
Called when a file has been refactored, and there are changes.
i Ns No changes to %ss Not writing changes to %s( R} R R! R R R t
write_file( R~ R R R R R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c C s y t | d d | } Wn, t i j
o } | i d | | d SXzI y | i t | Wn+ t i j
o } | i d | | n XWd | i X| i d | t | _ d S( s Writes a string to a file.
It first shows a unified diff between the old text and the new text, and
then rewrites the file; the latter is only done if the write option is
set.
t wR s Can't create %s: %sNs Can't write %s: %ss Wrote changes to %s(
R R
t errorR R RK R R RW Rv ( R~ R R R R R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s s >>> s ... c
C s g } d } d } d } d } x5| i t D]$} | d 7} | i i | i o_ | d j o# | i | i | | | | n | } | g } | i | i } | | } q. | d j oF | i | | i p | | | i i
d j o | i | q. | d j o# | i | i | | | | n d } d } | i | q. W| d j o# | i | i | | | | n d i | S( s Refactors a docstring, looking for doctests.
This returns a modified version of the input string. It looks
for doctests, which start with a ">>>" prompt, and may be
continued with "..." prompts, as long as the "..." is indented
the same as the ">>>".
(Unfortunately we can't use the doctest module's parser,
since, like most parsers, it is not geared towards preserving
the original source.)
i i u
u N(
R! t
splitlinesRW t lstripR t PS1R6 t refactor_doctestt findt PS2t rstripR R (
R~ RH R t resultt blockt block_linenot indentt linenot linet i( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s<
c C sb y | i | | | } Wny t j
om } | i i t i o. x+ | D] } | i d | i d qI Wn | i d | | | i i
| | SX| i | | o t | i
t } | | d | | d } } | d i d p | d c d 7>>" (possibly indented), while the remaining lines start
with "..." (identically indented).
s
Source: %su
s+ Can't parse docstring in %s line %s: %s: %si ii ( t parse_blockR( t logt isEnabledForRs t DEBUGR R R R R R R R RW R R t popR ( R~ R R R R R R R R t clippedRB ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s& ! 8c C s' | i o
d } n d } | i p | i d | n2 | i d | x | i D] } | i | qR W| i o2 | i d x" | i D] } | i | q Wn | i ot t | i d j o | i d n | i d t | i x1 | i D]" \ } } } | i | | | q Wn d S( Nt weres
need to bes No files %s modified.s Files that %s modified:s$ Warnings/messages while refactoring:i s There was 1 error:s There were %d errors:( Rv R} R Ru Rr R ( R~ R t filet messageR R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt summarize s*
c C s1 | i i | i | | | } t | _ | S( s Parses a block into a tree.
This is necessary to get correct line number / offset information
in the parser diagnostics and embedded into the parse tree.
( R t parse_tokenst wrap_toksRS R ( R~ R R R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR 1 s ! c c s t i | i | | i } xe | D]] \ } } \ } } \ } }
} | | d 7} | | d 7} | | | | f | |
f | f Vq% Wd S( s; Wraps a tokenize stream to systematically modify start/end.i N( R RP t gen_linesRL ( R~ R R R R5 R Rb t line0t col0t line1t col1t line_text( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR ; s !c c s | | i } | | i } | } xm | D]e } | i | o | t | Vn7 | | i d j o d Vn t d | | f | } q' Wx t o d Vq Wd S( s Generates lines as expected by tokenize from a list of lines.
This strips the first len(indent + self.PS1) characters off each line.
u
s line=%r, prefix=%rR N( R R R R R t AssertionErrorRW ( R~ R R t prefix1t prefix2t prefixR ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR I s
N( R R R@ Rm R R R! R Rx R R R R R R R R R R R R R R R R R R R R R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyRf s8
( +
t MultiprocessingUnsupportedc B s e Z RS( ( R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR ] s t MultiprocessRefactoringToolc B s5 e Z d Z e e d d Z d Z d Z RS( c O s/ t t | i | | d | _ d | _ d S( N( t superR R R! t queuet output_lock( R~ R t kwargs( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR c s i c
C sv | d j o t t | i | | | Sy d d k } Wn t j
o
t n X| i d j o t d n | i | _ | i
| _ g } t | D] } | | i
d | i q ~ } z; x | D] } | i q Wt t | i | | | Wd | i i x$ t | D] } | i i d q"Wx) | D]! } | i o | i qCqCWd | _ Xd S( Ni is already doing multiple processest target( R R R t multiprocessingt ImportErrorR R R! t RuntimeErrort
JoinableQueuet LockR
t xranget Processt _childt startR t putt is_alive(
R~ R R R t
num_processesR
RB R t processesR+ ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR h s8
/
c C sq | i i } x[ | d j oM | \ } } z t t | i | | Wd | i i X| i i } q Wd S( N( R t getR! R R R t task_done( R~ t taskR R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s
c O sG | i d j o | i i | | f n t t | i | | Sd S( N( R R! R R R R ( R~ R R ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR s ( R R R R@ R R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyR a s
(' Re t
__future__R t
__author__R
R Rs R R- RQ t itertoolsR t pgen2R R R R R R RW R R( R R% R> RD RF t version_infot codecsR R RI RK Rc Rd t objectRf R R ( ( ( s( /usr/lib64/python2.6/lib2to3/refactor.pyt s>
(