Proxy parser python github

Saved searches

Use saved searches to filter your results more quickly

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Free proxy scraper written in python. It is pypi library — free to use.

License

jundymek/free-proxy

This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Git stats

Files

Failed to load latest commit information.

README.md

FreeProxy class scrapes proxies from https://www.sslproxies.org/, https://www.us-proxy.org/, https://free-proxy-list.net/uk-proxy.html and https://free-proxy-list.net and checks if proxy is working. There is possibility to filter proxies by country and acceptable timeout. You can also randomize list of proxies from where script would get first working proxy.

You can use it in sending request through custom proxy, with selenium or wherever you want.

First import Free Proxy that way:

from fp.fp import FreeProxy

• country_id parameter Get first working proxy from specified list of countries (from https://www.sslproxies.org/). If there is no valid proxy from specified list check all countries from https://free-proxy-list.net.

proxy = FreeProxy(country_id=['US', 'BR']).get()

• country_id for US and GB You can set country_id to US and GB to get proxy from United States or United Kingdom. In that case proxies will be scrapped from https://www.us-proxy.org/ ( US ) or https://free-proxy-list.net/uk-proxy.html ( GB ) page. If there is no valid proxy from specified list check all countries

proxy = FreeProxy(country_id=['US']).get() proxy = FreeProxy(country_id=['GB']).get()

• timeout parameter Timeout is parameter for checking if proxy is valid. If test site doesn’t respond in specified time script marks this proxy as invalid. Default timeout=0.5 . You can change it by defining specified timeout eg. timeout=1 .

proxy = FreeProxy(timeout=1).get()

• rand parameter Shuffles proxy list from https://www.sslproxies.org/. Default rand=False and searches for working proxy from newest to oldest (as they are listed in https://www.sslproxies.org/).

proxy = FreeProxy(rand=True).get()

• anonym parameter Return only those proxies that are marked as anonymous. Defaults to anonym=False

proxy = FreeProxy(anonym=True).get()

• elite parameter Return only those proxies that are marked as ‘elite proxy’. Defaults to elite=False .

proxy = FreeProxy(elite=True).get()

Note that elite proxies are anonymous at the same time, thus anonym=True automatically when elite=True .

• google parameter If True it returns only those proxies that are marked as google, if False — as no google. Defaults to google=None that returns all proxies.

proxy = FreeProxy(google=True).get()

• https parameter If true it returns only those proxies that are marked as HTTPS. Defaults to https=False — i.e. HTTP proxy (for HTTP websites). Note that HTTPS proxy is for both HTTP and HTTPS websites.

proxy = FreeProxy(https=True).get()

You can combine parameters:

proxy = FreeProxy(country_id=['US', 'BR'], timeout=0.3, rand=True).get()

If there are no working proxies with provided parameters script raises FreeProxyException with There are no working proxies at this time. message.

• Added new website to get proxies from https://free-proxy-list.net
• Added new website to get proxies from https://free-proxy-list.net/uk-proxy.html
• Added new website to get proxies from https://www.us-proxy.org/
• Change lxml version to 4.9.1

• Added elite parameter
• Add exception class and raise exception instead of system exit
• Change lxml version to 4.6.5

• Fix typos in readme
• Fix urrlib3 exception urllib3.exceptions.ProxySchemeUnknown: Not supported proxy scheme None ,
• Fix imports

Free Software!

About

Free proxy scraper written in python. It is pypi library — free to use.

Источник

Saved searches

Use saved searches to filter your results more quickly

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

License

bu77h4ad/proxy_parser

This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Git stats

Files

Failed to load latest commit information.

README.md

#пример использования import proxy_parser #добавляем парсер в свой проект a = proxy_parser.proxy_parser() #создание объекта класса a.get_proxies('https',1) #парсит прокси с сайта в объект класса #первый аргумент тип прокси - https,http,socks4,socks5,socks,all. #следующий аргумент количество страниц с проксями из сайта(обычно 35 прокси из 1 страницы). print (a.server[1]['ip'], a.server[1]['port'], a.server[1]['protocol'], len(a.server), a.next_proxy()) #вывод будет таким #200.73.132.107 3129 HTTPS 35

Источник

Saved searches

Use saved searches to filter your results more quickly

You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session. You switched accounts on another tab or window. Reload to refresh your session.

Easy-to-use sync library for handy proxy parse

License

krilifon/proxy-parse

This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.

Name already in use

A tag already exists with the provided branch name. Many Git commands accept both tag and branch names, so creating this branch may cause unexpected behavior. Are you sure you want to create this branch?

Sign In Required

Please sign in to use Codespaces.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching GitHub Desktop

If nothing happens, download GitHub Desktop and try again.

Launching Xcode

If nothing happens, download Xcode and try again.

Launching Visual Studio Code

Your codespace will open once ready.

There was a problem preparing your codespace, please try again.

Latest commit

Git stats

Files

Failed to load latest commit information.

README.md

Synchronous library, for convenient and fast parsing of proxies from different sources.

At the moment the library does not support automatic proxy check, this option will be added in the asynchronous version of the library.

Installing the latest version of the library:

from proxy_parse import ProxyParser proxy_parser = ProxyParser() proxies_list = proxy_parser.parse()

If you need, you can add some parameters to the ProxyParser class:

• path_to_file — optional str parameter, the proxies will be saved to a file at the path
• proxy_limit — optional int parameter, the ProxyParser.parse function will return as many proxies as you need
• scrapy_spiders — optional scrapy.Spider list parameter, you can add your own spiders, which will work together with the others
• scrapy_settings — optional dict parameter, you can replace the library rules with your own

Any changes from you will be good for the life of the library

About

Easy-to-use sync library for handy proxy parse

Источник