Whl自助搜索下载器

Whl自助搜索下载器

本文转载自以下链接：https://github.com/Light-City/AutoDownloadWhl

源码地址：

https://github.com/Light-City/AutoDownloadWhl

本次脚本我将它命名为：Whl自助搜索下载器

实现了如下功能：

• 支持检索
• 支持查找
• 支持下载
• 单、多文件下载

2.如何使用及功能演示 2.1 如何使用

运行脚本需要：

• requests、lxml、os库
• curl或axel命令

win系统的直接下载win版本的按照网上配置即可！

linux系统与mac系统可用yum 或 apt 或 brew等来进行安装！

复制源码地址并下载脚本，运行脚本即可：

python AutoDownloadWhl.py

2.2 功能演示

• 所有包信息展示
• 适应平台whl包
• 多包下载

3.代码详解 3.1 封装

封装包地址与下载地址：

class AutoDownWhl():

    def __init__(self):

        # 包地址

        self.url = 'https://www.lfd.uci.edu/~gohlke/pythonlibs/'

        # 下载地址

        self.base_url = 'https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/'

        # 模拟浏览器

        self.headers = {

            'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3626.96 Safari/537.36'

        }

3.2 获取选择器与包名

解析源码与获取包名：

# 获取选择器与包名

def getLib(self):

    raw_text = requests.get(self.url, headers=self.headers).content

    seletor = etree.HTML(raw_text)

    lib_names = seletor.xpath('//ul[@class="pylibs"]//li//strong//text()')

    return seletor, lib_names

3.3 输出所有包名

一行15个包名打印：

# 输出所有包名

    def print_AllLib(self, lib_names):

        print('\n------------------共查找到' + str(len(lib_names)) + '个包------------------\n')

        for i in range(len(lib_names)):

            if i and i % 15 == 0:

                print('\n')

            print(lib_names[i], end=' ')

3.4 搜索与下载

只摘取了核心代码，下载采用调用系统命令curl与axel，请在自己的系统上安装这两个其中之一下载！

def searchDown(self, downloadMethod='axel'):

    # 此部分代码只摘取核心部分

    if int(lib_sequence) <= max:

        detail_name = detail_libs[int(lib_sequence) - 1].strip().replace('‑', '-')

        download_url = self.base_url + detail_name

        print('\n\n输入包名正确,下载地址：' + download_url + '\n')

        print("---------------------------------------开始下载---------------------------------------\n")

        if downloadMethod == 'curl':

            cmd = 'curl -O %s' % download_url

        else:

            cmd = 'axel %s' % download_url

        while True:

            cmd_res = os.system(cmd)

            if cmd_res == 0:

                print('下载成功!\n')

                break

            else:

                print('下载失败!\n')

                is_try = input('是否重试下载(y|n)： ')

                if is_try == 'y' or is_try == 'Y':

                    continue

                else:

                    break

    else:

        print("\n \033[1;31m 输入序号不符合要求,请重新输入！ \033[0m \n")

3.5 调用

采用axel下载：

dw = AutoDownWhl()

dw.searchDown(downloadMethod='axel')

实际运行效果显示：
因为下载使用的windows系统版本的axel命令无法访问https，
(使用axel命令下载报错，错误原因是Unable to connect to server download.lfd.uci.edu:80。

使用浏览器直接访问download.lfd.uci.edu报错，但是通过查看包的下载地址，地址开头是https，使用https://www.lfd.uci.edu/就能访问该网站，而http://www.lfd.uci.edu/却无法访问。

)

所以使用curl命令，修改最后一行：dw.searchDown(downloadMethod='curl')  (前提是需要把curl命令添加到系统环境变量里)

执行python AutoDownloadWhl.py
首先会列出当前搜索到的所有whl包，接下来输入下载包的名字(英文逗号分割)，然后会按照顺序，挨个查找下载包，根据python版本的不同，输入需要下载的包序号，会出现该包的下载地址。


然后curl就开始下载包了，下载的包会保存到当前目录。



或者只获取包的下载地址，使用迅雷下载

完整的下载演示如下所示：

C:\Users\sandu\Anaconda3\envs\Network\python.exe D:/scrapyseleniumtest/AutoDownloadWhl.py

---------------------------------------欢迎使用Whl自助搜索下载器---------------------------------------

------------------共查找到476个包------------------

AD3 Aggdraw Aiohttp Akima APSW Arctic Aspell-python Assimulo Astropy Atom AutoPy Babel Backports.lzma Basemap Bazaar 

Bcolz Bigfloat Bintrees Bio_formats Biopython Bitarray Blaze Blist Blosc Blz Boost.Python Bottleneck Brotli brotlipy Bsddb3 

Bsdiff4 BTrees Cairocffi Cantera Carray Cartopy Casuarius cChardet Cdecimal CellCognition CellProfiler Centrosome ceODBC Cf-units Cffi 

Cftime CGAL-bindings Cgkit Chaco Chebyfit Cheetah Chompack CLD Cmapfile cmarkgfm COBRA Coverage Crc16 Crcmod Cupy 

Curses CVXcanon CVXOPT CVXPY cx_Freeze cx_Oracle Cyassimp Cyordereddict Cyrasterize Cython Cytoolz Czifile Daal4py Datrie Debug-information-files 

Delny Dipy Discretize Dnacurve Dulwich DyND ECOS Enable Enaml Entropy ETS FabIO Fann2 Fast-histogram Fastcache 

Fastparquet Fastrlock fastText fastTSNE Faulthandler Fcsfiles Fdint Ffnet Fiona Fisher Fisx FLANN Fmkr Fpzip FreeImageDLL 

FreeSASA Friture GDAL Genshi Gensim Gevent Glumpy GMPY GPy GR Grako Greenlet gRPCio GuiQwt Gvar 

H5py HDDM Heatmap Hmmlearn HoloPy HTSeq HyperSpy Ilastik Imagecodecs ImageD11 Iminuit Imread Imreg Indexed_gzip Intbitset 

IOCBio Iris Javabridge JCC JPype Jsonlib Jupyter Kapteyn Kivy Kiwisolver Kwant La Lazy_object_proxy Lazyflow Lfdfiles 

LibLAS LIBLINEAR Libpython libSBML LIBSVM Libtfr Libxml-python Lightning Line_profiler Llist Llvmlite LLVMPy Logbook Louvain-igraph lp_solve 

Lru_dict Lsqfit Lucam Lxml Lz4 Mahotas Marisa-trie MarkupSafe Mathutils Matplotlib Mayavi Menpo Mercurial MeshPy Milk 

Minepy Mistune MKL-service MKL_fft MKL_random Mlpy mmLib MMTK Mod_wsgi ModernGL Molmass Mpi4py Mplcairo Msgpack Multidict 

MultiNEAT Multiprocess mxBase MySQL-python Mysqlclient Natgrid NDimage NetCDF4 Netifaces Netpbmfile NIPY NiTime NLopt nMOLDYN Noise 

Numba Numcodecs Numexpr NumPy Numpy-quaternion numpy-stl ObsPy ODE Oiffile Open3D OpenBabel OpenCV OpenEXR OpenImageIO OpenPIV 

openTSNE Orange Osqp Oursql Pandas Param Peewee Pendulum Persistent Pgmagick Pillow Pillow-SIMD Pip Planar Pocketsphinx 

Polygon Polylearn Polymode PostgresAdapter Protobuf Psf Psutil Psycopg PuLP Py-earth Py-fcm Py-lmdb Py2exe Py_gd PyAlembic 

PyAMF PyAMG PyArrow Pyasn PyAudio PyBluez PyBox2D Pycairo Pycares PyCifRW PyCLD2 PyCluster PyCogent PyCorrFit Pycosat 

PyCryptoSat PyCUDA PycURL Pycvodes PyDbg PyDDE PyEDA PyEMD PyEphem Pyexiv2 PyFFTW PyFITS pyFLTK PyFlux pyFM 

PyFMI Pygame Pygit2 PyGraphviz PyGreSQL PyGTK PyHDF PyHook Pyhull PyICU PyISAPIe PyJNIus Pykinsol PyKSVD Pyldap 

PyLibDeconv PyLibTiff PyLZMA Pymatgen PyMC PyMca PyMCMC PyMCubes PyMedia PyMetis PyMinuit PyMix PyMOL Pymongo Pymssql 

Pymunk Pymutt PyMVPA PyODBC Pyodeint PyOpenCL PyOpenGL Pypmc Pyproj PyQPBO PyQt4 PyQwt Pyropes Pyrsistent pyRXP 

PySFML PySide pySLALIB PySparse Pyspharm Pysqlite PyStemmer PyStruct Pyswisseph PyTables Python-cjson Python-geohash Python-igraph Python-ldap Python-Levenshtein 

Python-lzf Python-LZO Python-snappy Python-sundials PythonMagick PythonNET Pytiff PyTorch PyTST PyUSB-FTDI PyViennaCL PyVRML97 PyWavelets PyWCS PyWin32 

PyWinHook Pywinpty PyX PyXML PyYAML PyZMQ Pyzopfli Qdafile Qimage2ndarray Qt_graph_helpers QuantLib QuickFIX QuTiP Rapidjson Rasterio 

Re2 Recordclass Regex ReportLab Rpy2 Rtmidi-python Rtree Ruamel.yaml Salientdetect Sasl Scandir ScientificPython Scikit-cycling Scikit-fmm Scikit-image 

Scikit-learn Scikit-misc Scikit-umfpack Scikits.ann Scikits.audiolab Scikits.delaunay Scikits.hydroclimpy Scikits.odes Scikits.samplerate Scikits.scattpy Scikits.timeseries Scikits.vectorplot Scimath SciPy Scipy-cluster 

SCS Sdtfile SendKeys Seqlearn Setproctitle SfePy Shapely SilverCity Sima Simpleaudio SimpleITK Simplejson SimpleParse Slycot Sounddevice 

spaCy Sparsesvd Spectrum Spglib SQLAlchemy Statsmodels STEPS Stratify Swiglpk Sympy TA-Lib TatSu TensorFlow Thrift ThriftPy 

Tifffile Tinyarray Tomopy Tornado Traits Transformations Triangle Trollius TWAINmodule Twisted typed_ast Uciwebauth UDUNITS Ujson Umysql 

Veusz VideoCapture Vidsrc VIGRA ViTables VPython VTK Winrandom Wordcloud Wrapt Wrf_python wxPython x86cpu Xgboost Xxhash 

XYlib-py Yappi Yarl Yt Zfec Zipline Zodbpickle Zope.interface Zs Zstd Misc 

请输入下载包的名字(英文逗号分割): SQLAlchemy

---------查找SQLAlchemy成功,正在输出包的详细信息---------

(1)SQLAlchemy‑1.2.18‑cp27‑cp27m‑win32.whl

(2)SQLAlchemy‑1.2.18‑cp27‑cp27m‑win_amd64.whl

(3)SQLAlchemy‑1.2.18‑cp35‑cp35m‑win32.whl

(4)SQLAlchemy‑1.2.18‑cp35‑cp35m‑win_amd64.whl

(5)SQLAlchemy‑1.2.18‑cp36‑cp36m‑win32.whl

(6)SQLAlchemy‑1.2.18‑cp36‑cp36m‑win_amd64.whl

(7)SQLAlchemy‑1.2.18‑cp37‑cp37m‑win32.whl

(8)SQLAlchemy‑1.2.18‑cp37‑cp37m‑win_amd64.whl

请输入下载的包名序号： 8

输入包名正确,下载地址：https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/SQLAlchemy-1.2.18-cp37-cp37m-win_amd64.whl

---------------------------------------开始下载---------------------------------------

curl -O https://download.lfd.uci.edu/pythonlibs/r5uhg2lo/SQLAlchemy-1.2.18-cp37-cp37m-win_amd64.whl

  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current

                                 Dload  Upload   Total   Spent    Left  Speed

100 1087k  100 1087k    0     0  39631      0  0:00:28  0:00:28 --:--:-- 45299

下载成功!

是否继续下载其他库(y|n)： n

---------------------------------------感谢使用---------------------------------------

修改后使用curl命令下载的代码地址：

https://files.cnblogs.com/files/sanduzxcvbnm/AutoDownloadWhl-master.7z