gwddlZddlZddlZddlZddlmZddlZddlm Z ddl Zddl m Z m Z mZmZmZmZmZmZddlmZddlmZddlmZmZej6d k\r dd lmZmZmZn dd l m!ZmZmZ dd l"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(dd l)m*Z*ej\e/Z0dZ1dZ2dZ3dZ4dgZ5ddddddddZ6GddZ7y#e+$rdd l,m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(dd l-m*Z*YOwxYw)N) namedtuple)sleep)IPDefinedErrorASNLookupErrorBlacklistErrorWhoisLookupErrorHTTPLookupErrorHostLookupErrorHTTPRateLimitErrorWhoisRateLimitError) RIR_WHOIS)ASN_ORIGIN_WHOIS)ipv4_is_definedipv6_is_defined)r) ip_address IPv4Address IPv6Address) IPAddressrr)OpenerDirector ProxyHandler build_openerRequestURLError HTTPError) urlencodezDhttp://whois.arin.net/rest/nets;q={0}?showDetails=true&showARIN=truezwhois.cymru.comz{0}.origin.asn.cymru.comz{0}.origin6.asn.cymru.comzroot.rwhois.netarinripenccapniclacnicafrinic)ARINzVR-ARINRIPEAPNICLACNICAFRINICDNICcneZdZdZd dZdZddZddZddZ ddZ dd Z dd Z dd Z dd Z y)Neta  The class for performing network queries. Args: address (:obj:`str`/:obj:`int`/:obj:`IPv4Address`/:obj:`IPv6Address`): An IPv4 or IPv6 address timeout (:obj:`int`): The default timeout for socket connections in seconds. Defaults to 5. proxy_opener (:obj:`urllib.request.OpenerDirector`): The request for proxy support. Defaults to None. Raises: IPDefinedError: The address provided is defined (does not need to be resolved). Nct|tst|tr||_nt ||_||_t jj|_ t|jdrt|jd|_ nt|jd|_ ||j_||j_ t|tr||_nt!}t#||_|jj%|_|jj(|_|j(dk(rt+|j&}|dr,t-dj/|j&|d|d|j&j1d}|j3dj5||_t8j/|j6|_yt=|j&}|dr,t-d j/|j&|d|d|jj>}|j1d }t7tAtC|D]\} } | d k(r|| =nd j5|}tE|jGd d } | ddd } dj5| |_tHj/|j6|_y)Nresolvequeryrz3IPv4 address {0} is already defined as {1} via {2}.r.z3IPv6 address {0} is already defined as {1} via {2}.:0000)% isinstancerraddressrtimeoutdnsresolverResolver dns_resolverhasattrgetattr dns_resolvelifetimeropenerrr__str__ address_strversionrrformatsplitreversejoinreversed IPV4_DNS_ZONEdns_zonerexplodedlist enumeratestrreplace IPV6_DNS_ZONE) selfr6r7 proxy_openerhandler is_definedrErKgroupsindexvaluevals /opt/Tautulli/lib/ipwhois/net.py__init__z Net.__init__ps g{ +z0&#DL &g.DL LL113 4$$i 0&t'8'8)DD &t'8'8'BD $+!%," lN 3&DK#nG&w/DK <<//1||++ <<1 ))9)9:J!}$!6((*Q-A$$**3/E MMOHHUODM)00?DM ))9)9:J!}$!6((*Q-A||,,H^^C(F (i.?)@ A  uF?u  xx'Hh-''R0Cdd)CHHSMDM)00?DMcJ tjdj|j|j |jd}t |S#t jjt jjt jjt jjf$r>}tdj|jj|j d}~wtdj|j xYw)z The function for retrieving ASN information for an IP address from Cymru via port 53 (DNS). Returns: list: The raw ASN data. Raises: ASNLookupError: The ASN lookup failed. ASN query for {0}TXT$ASN lookup failed (DNS {0}) for {1}.NASN lookup failed for {0}.)logdebugrDrJr>rLr8r9NXDOMAIN NoNameserversNoAnswer exceptionTimeoutr __class____name__rB)rQdataes rY get_asn_dnszNet.get_asn_dnss  II)00? @##DMM59D:  %%s||'A'A %%s}}'<'<> !6==KK(($*:*:<    ,33D4D4DE sAAA)D"9C99)D"cT|dddk7rdj|}dj|} tjdj||j|d}t |dS#t j jt j jt j jt jjf$r4}tdj|jj|d }~wtd j|xYw) a The function for retrieving the information for an ASN from Cymru via port 53 (DNS). This is needed since IP to ASN mapping via Cymru DNS does not return the ASN Description like Cymru Whois does. Args: asn (:obj:`str`): The AS number (required). Returns: str: The raw ASN data. Raises: ASNLookupError: The ASN lookup failed. rr/ASzAS{0}z{0}.asn.cymru.comzASN verbose query for {0}r^r_Nr`)rDrarbr>rNr8r9rcrdrerfrgrrhri)rQasnzonerjrks rYget_asn_verbose_dnszNet.get_asn_verbose_dnss q8t ..%C"))#.  II188> ?##D%0DtAw<  %%s||'A'A %%s}}'<'<> !6==KK((#/    ,33C8 sAA00A)D'/DD'c tjtjtj}|j|jt j dj|j|jtdf|jdj|jdjd} |jdj}||z }|sn(|jt!|S#tjtj"f$r}t j dj||dkDrFt j d jt!||j%|d z cYd }~St'd j|jd }~wt'd j|jxYw) a The function for retrieving ASN information for an IP address from Cymru via port 43/tcp (WHOIS). Args: retry_count (:obj:`int`): The number of times to retry in case socket errors, timeouts, connection resets, etc. are encountered. Defaults to 3. Returns: str: The raw ASN data. Raises: ASNLookupError: The ASN lookup failed. r]+z -r -a -c -p -f {0}{1} r3ASN query socket error: {0}rASN query retrying (count: {0})rNr`)socketAF_INET SOCK_STREAM settimeoutr7rarbrDrBconnect CYMRU_WHOISsendencoderecvdecodecloserNerror get_asn_whoisr)rQ retry_countconnrjdrks rYrzNet.get_asn_whoiss"/ ==1C1CDD OODLL ) II)001A1AB C LL+r* + II(//$$f.fh  DIIdO**,  JJLt9  -  II3::1= >Q ;BB $&'))+/::%0778H8HI  ,33D4D4DE s%DD#G26A)G G2%$G  )G2c tjdj|j|j t t j|j|ddi}|S#tjtjf$r}tjdj||dkDrGtjdjt ||j|dz cYd }~Std j|jd }~wtd j|jxYw) a The function for retrieving ASN information for an IP address from Arin via port 80 (HTTP). Currently limited to fetching asn_registry through a Arin whois (REST) lookup. The other values are returned as None to keep a consistent dict output. This should be used as a last chance fallback call behind ASN DNS & ASN Whois lookups. Args: retry_count (:obj:`int`): The number of times to retry in case socket errors, timeouts, connection resets, etc. are encountered. Defaults to 3. Returns: dict: The ASN data in json format. Raises: ASNLookupError: The ASN lookup failed. r]Acceptzapplication/json)urlrheadersrvrrwr)rNr`) rarbrDrB get_http_jsonrNr#rxr7r get_asn_httpr)rQrresponserks rYrzNet.get_asn_httpas!(  II)001A1AB C))I$$T%5%56'!#56*H O -  II3::1= >Q ;BB $&'(([](CC%0778H8HI  ,33D4D4DE s%A+A..#EA*D%;E$D%%)Ec | t|d}tjtjtj}|j |j t jdj||||j||fdj|d}|j|jd} |jdj} || z }| sn(|jd|vrX|d kDr9t jd td |j!|||d z || St#d j|d|vsd|vr*t jdj|t$t'|S#tj tj(f$r} t jdj| |d kDrKt jdjt'||j!|||d z || cYd} ~ St+dj|d} ~ wt"$rt+dj|xYw)aa The function for retrieving CIDR info for an ASN via whois. Args: asn_registry (:obj:`str`): The source to run the query against (asn.ASN_ORIGIN_WHOIS). asn (:obj:`str`): The AS number (required). retry_count (:obj:`int`): The number of times to retry in case socket errors, timeouts, connection resets, etc. are encountered. Defaults to 3. server (:obj:`str`): An optional server to connect to. port (:obj:`int`): The network port to connect on. Defaults to 43. Returns: str: The raw ASN origin whois data. Raises: WhoisLookupError: The ASN origin whois lookup failed. WhoisRateLimitError: The ASN origin Whois request rate limited and retries were exhausted. Nserverz)ASN origin WHOIS query for {0} at {1}:{2}z -i origin {0}{1}rtr3ruQuery rate limit exceededrz6ASN origin WHOIS query rate limit exceeded. Waiting...r) asn_registryrorrportzmASN origin Whois lookup failed for {0}. Rate limit exceeded, wait and try again (possibly a temporary block). error 501 error 230z!ASN origin WHOIS query error: {0}z(ASN origin WHOIS query socket error: {0}z,ASN origin WHOIS query retrying (count: {0})z'ASN origin WHOIS lookup failed for {0}.)rrxryrzr{r7rarbrDr|r~rrrrrget_asn_origin_whoisr ValueErrorrNrr ) rQrrorrrrr-rrrks rYrzNet.get_asn_origin_whoissQ0V ~),7A==1C1CDD OODLL ) IIAHHVT# $ LL&$ ((..sF;E IIelln %HIIdO**,A  JJL+h6?II+,!H44%1s$/M%D5.,,2F3K99 )[H-D =DDXNO  x=  -  II@GGJ KQ #VC $45700!-3 +A f41'=DDSI#   "9@@E s,EFAF#I4>A.I ,I42I  (I4c |r|ng}tfdt|fDrtdj t|dt j t j t j}|j|jtjdj|j||j|f|jdz}|dk(rdj|}|j|jd } |j!d j#d d } | | z } | sn*|j%d | vrb|dkDr9tjdt'd|j)||dz ||St+dj|jd| vsd| vr*tjdj| t,t/| S#t jt j0f$r} tjdj| |dkDrKtjdjt/||j)||dz ||cYd} ~ St3dj|jd} ~ wt*$rt$rt3dj|jxYw)a The function for retrieving whois or rwhois information for an IP address via any port. Defaults to port 43/tcp (WHOIS). Args: asn_registry (:obj:`str`): The NIC to run the query against. Defaults to 'arin'. retry_count (:obj:`int`): The number of times to retry in case socket errors, timeouts, connection resets, etc. are encountered. Defaults to 3. server (:obj:`str`): An optional server to connect to. If provided, asn_registry will be ignored. port (:obj:`int`): The network port to connect on. Defaults to 43. extra_blacklist (:obj:`list` of :obj:`str`): Blacklisted whois servers in addition to the global BLACKLIST. Defaults to None. Returns: str: The raw whois data. Raises: BlacklistError: Raised if the whois server provided is in the global BLACKLIST or extra_blacklist. WhoisLookupError: The whois lookup failed. WhoisRateLimitError: The Whois request rate limited and retries were exhausted. c3&K|]}|v ywN).0srvrs rY z Net.get_whois..(sBS6S=BszThe server {0} is blacklisted.NrzWHOIS query for {0} at {1}:{2}rtrzn + {0}r3ruasciiignorerrz+WHOIS query rate limit exceeded. Waiting...r)rrrrextra_blacklistzbWhois lookup failed for {0}. Rate limit exceeded, wait and try again (possibly a temporary block).rrzWHOIS query error: {0}zWHOIS query socket error: {0}z!WHOIS query retrying (count: {0})zWHOIS lookup failed for {0}.)any BLACKLISTrrDrrxryrzr{r7rarbrBr|r~rrrrr get_whoisr rrNrr ) rQrrrrrextra_blrr-rrrks ` rYrz Net.get_whoiss:b *9rHBY,ABB$4;;FC~"<0:==1C1CDD OODLL ) II6==  &$0 1 LL&$ ($$v-Ev%!((/ IIelln %HIIdO**7H=A  JJL*h6?IIKL!H>>%1{1}%D(7*.,,2F43C3C,DFF )[H-D 299(CD  x=  -  II5<&E&E #$6 7'9:""45#'#5#5$'[]3E(/$6$ #5!44:F3K#AAA0H=" FJJtyy{11'8DE F4j) H    Lvv} ;<?II6==./12,---[1_+= '. -,,2F3K99&'228&c!&&k2JLL&..&,,7  II4;;A> ?Q <CC $&'))Q'97*&&C&J&J' !34 G M!"?"F"Fs"KL LsA#F,3D; E=1A+E=E=9F; ?Q <CC $&'}}[1_55&1889I9IJ !-44T5E5EF s%B+B..#F A)E$:F $E$$)F c|ddi}d}|rt|} t|d} tj dj |j | td|||dd|i}|jj||j } |jjdd } t| S#t$rYwxYw#t$rt|||}Y{wxYw#t$r#|jjdd } YbwxYw#t t"jt"j$f$r} tj d j | |d kDrKtj d j t||j'||dz |||cYd} ~ St)dj |d} ~ wt($r} | d} ~ wt*$rt)dj |wxYw)a The function for retrieving a raw HTML result via HTTP. Args: url (:obj:`str`): The URL to retrieve (required). retry_count (:obj:`int`): The number of times to retry in case socket errors, timeouts, connection resets, etc. are encountered. Defaults to 3. headers (:obj:`dict`): The HTTP headers. The Accept header defaults to 'text/html'. request_type (:obj:`str`): Request type 'GET' or 'POST'. Defaults to 'GET'. form_data (:obj:`dict`): Optional form POST data. Returns: str: The raw data. Raises: HTTPLookupError: The HTTP lookup failed. Nrz text/htmlr)encodingr)rrjrmethodrrrrrr)rrr request_type form_datarr)rbytes TypeErrorrarbrDrBrr@rr7rrrrrNrrxr get_http_rawr Exception) rQrrrrr enc_form_datarrjrrks rYrzNet.get_http_rawAs . ?-G %i0M  %mg F , M II188  #' ( M;3]G;"*L!9;;;##D$,,#?D :LLN))'8<q6M+   M3]GL M " :IIK&&w9 : &..&,,7  II4;;A> ?Q <CC $&'((q'!-)&&C&J&J' G M!"?"F"Fs"KL L Ms B9/DC&'D C%. D9 CCC"D!C""D%)DDDD(G?@n`DH+/BIL"yMv<|=A37OMr[r*)8sysrx dns.resolverr8r collectionsrloggingtimerdns.rdtypes.ANY.TXT exceptionsrrrr r r r r whoisrrorutilsrr version_info ipaddressrrripaddrrurllib.requestrrrrrr urllib.parser ImportErrorurllib2urllib getLoggerrirar#r}rIrPrORG_MAPr*rr[rYrs2  "BBB!3v((%%!++ 'g!N * +       q Mq MG!$$ !!s2B::CC