官方说明
Colaboratory 是一个研究项目，可免费使用。（暂不确定这个项目是不是长久有效）
Colaboratory 支持 Python 2.7 和 Python 3.6。
谷歌中文说明：https://colab.research.google.com/notebook
以下教程基于Google浏览器（桌面版 Chrome）操作，并且使用代理访问Google。
申请Colaboratory
1、进入谷歌云盘（ https://drive.google.com/drive/my-drive ），新建一个文件夹（本教程以app命名）
2、创建Colaboratory，进入创建好的文件夹，点开新建 -> 更多 ->关联更多应用 。搜索并找到【Colaboratory】，然后关联
3、创建新的Colab笔记本
通过右键单击>更多> Colaboratory创建一个新的jupyter笔记本
通过点击文件名来重命名。
使用Colaboratory
1、设置GPU运行，选择 修改 -> 设置
2、设置后端Python版本和免费的GPU使用
现在已经可以运行一些ubuntu 命令，每条命令以英文的【！】开始
3、新建代码与运行
①新建代码
②输入代码内容
③点击运行代码
根据自己的需求安装一些软件
4、使用Google Colab运行py文件，读取和存储到Google云盘中
首先运行这些代码，以便安装必要的库并执行授权。
|
1 2 3 4 5 6 7 8 9 10 11 12 |
!apt-get install -y -qq software-properties-common python-software-properties module-init-tools !add-apt-repository -y ppa:alessandro-strada/ppa 2>&1 > /dev/null !apt-get update -qq 2>&1 > /dev/null !apt-get -y install -qq google-drive-ocamlfuse fuse from google.colab import auth auth.authenticate_user() from oauth2client.client import GoogleCredentials creds = GoogleCredentials.get_application_default() import getpass !google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} < /dev/null 2>&1 | grep URL vcode = getpass.getpass() !echo {vcode} | google-drive-ocamlfuse -headless -id={creds.client_id} -secret={creds.client_secret} |
注:!pip install或者 !apt-get install命令可以安装其他能使用到的库。
运行可以看到如下结果：此时，点击链接地址，获取验证码。
点击链接地址，获取授权码，把获取到的授权码输入之后回车出现如下图所示，再点击这个链接又会获取Google Cloud SDK，操作和之前一样。
出现下面提示说明已经授权成功
Please enter the verification code: Access token retrieved correctly.
补充：如果出现如下提示
Warning: apt-key output should not be parsed (stdout is not a terminal)
这说明认证已经完成，直接执行一步就可以了。
5、挂载Google Drive
|
1 2 |
!mkdir -p drive !google-drive-ocamlfuse drive |
挂载完成后就可以使用Google云盘中的文件了，（因为这个jupyter笔记本设置的是python3所以下面事例用python3）
|
1 |
!python3 drive/app/xxx.py |
6、更改工作目录
查看工作目录
你能看到datalab和驱动器文件夹。因此，您必须在定义每个文件名之前添加drive/app。为了避免这个麻烦，我们可以更改工作目录
更改工作目录为app（你自己使用哪个文件，改下面命令中的名字就可以，禁止使用中文。）
|
1 2 |
import os os.chdir("drive/app") |
相关命令
1、查看是否使用GPU
|
1 2 |
import tensorflow as tf tf.test.gpu_device_name() |
2、在使用哪个GPU
|
1 2 |
from tensorflow.python.client import device_lib device_lib.list_local_devices() |
3、RAM信息
|
1 |
!cat /proc/meminfo |
4、CPU信息
|
1 |
!cat /proc/cpuinfo |
91福利python3脚本
把代码保存为.py文件格式
1、推荐使用（服务器&Colaboratory都可以使用）
运行脚本前安装requests库：
|
1 |
!pip install requests |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 |
import requests import os,re,time,random def download_mp4(url,dir): headers={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36Name','Referer':'http://91porn.com'} req=requests.get(url=url) filename=str(dir)+'/1.mp4' with open(filename,'wb') as f: f.write(req.content) def download_img(url,dir): headers={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36Name','Referer':'http://91porn.com'} req=requests.get(url=url) with open(str(dir)+'/thumb.png','wb') as f: f.write(req.content) def random_ip(): a=random.randint(1,255) b=random.randint(1,255) c=random.randint(1,255) d=random.randint(1,255) return(str(a)+'.'+str(b)+'.'+str(c)+'.'+str(d)) flag=63 while flag<=4250: tittle=[] base_url='http://91porn.com/view_video.php?viewkey=' page_url='http://91porn.com/v.php?next=watch&page='+str(flag) get_page=requests.get(url=page_url) viewkey=re.findall(r'<a target=blank href="http://91porn.com/view_video.php?viewkey=(.*)&page=.*&viewtype=basic&category=.*?">n <img ',str(get_page.content,'utf-8',errors='ignore')) for key in viewkey: headers={'Accept-Language':'zh-CN,zh;q=0.9','User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36','X-Forwarded-For':random_ip(),'referer':page_url,'Content-Type': 'multipart/form-data; session_language=cn_CN'} video_url=[] img_url=[] base_req=requests.get(url=base_url+key,headers=headers) video_url=re.findall(r'<source src="(.*?)" type='video/mp4'>',str(base_req.content,'utf-8',errors='ignore')) tittle=re.findall(r'<div id="viewvideo-title">(.*?)</div>',str(base_req.content,'utf-8',errors='ignore'),re.S) img_url=re.findall(r'poster="(.*?)"',str(base_req.content,'utf-8',errors='ignore')) try: t=tittle[0] tittle[0]=t.replace('n','') t=tittle[0].replace(' ','') except IndexError: pass if os.path.exists(str(t))==False: try: os.makedirs(str(t)) print('开始下载:'+str(t)) download_img(str(img_url[0]),str(t)) download_mp4(str(video_url[0]),str(t)) print('下载完成') except: pass else: print('已存在文件夹,跳过') flag=flag+1 print('此页已下载完成，下一页是'+str(flag)) |
2、另类91爬虫
运行脚本前安装requests_html库：
|
1 2 |
!pip install requests !pip install requests_html |
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 |
# coding:utf-8 from requests_html import HTMLSession import os import random import requests import traceback ualist = [ "Mozilla/4.0 (compatible; MSIE 9.0; Windows NT 6.1)", "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko", "Mozilla/5.0 (Windows NT 6.1; Trident/7.0; rv:11.0) like Gecko", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)", "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko", "Mozilla/5.0 (Windows NT 6.2; WOW64; Trident/7.0; rv:11.0) like Gecko", "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; rv:11.0) like Gecko", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; Trident/5.0)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)", "Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0)", "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; Media Center PC 6.0; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET CLR 2.0.50727; .NET4.0C; .NET4.0E)", "Mozilla/5.0 (Windows NT 6.3; WOW64; Trident/7.0; Touch; rv:11.0) like Gecko", "Mozilla/5.0 (Windows NT 6.2; Trident/7.0; rv:11.0) like Gecko", "Mozilla/5.0 (Windows NT 6.2; Win64; x64; Trident/7.0; rv:11.0) like Gecko", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.0; WOW64; Trident/5.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Win64; x64; Trident/5.0)", "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; MDDSJS; rv:11.0) like Gecko", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)", "Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; AS; rv:11.0) like Gecko", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; Trident/6.0) LinkCheck by Siteimprove.com", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/7.0)", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)", "Mozilla/5.0 (Windows NT 6.3; ARM; Trident/7.0; Touch; rv:11.0) like Gecko", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729)", "Mozilla/5.0 (Windows NT 10.0; Trident/7.0; rv:11.0) like Gecko", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Trident/6.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; Xbox)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)", "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.2; WOW64; Trident/7.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.2; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729)", "Mozilla/5.0 (Windows NT 6.0; WOW64; Trident/7.0; rv:11.0) like Gecko", "Mozilla/5.0 (Windows NT 5.1; WOW64; Trident/7.0; rv:11.0) like Gecko", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)", "Mozilla/5.0 (Windows NT 6.3; Win64; x64; Trident/7.0; rv:11.0) like Gecko", "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E; InfoPath.3)", "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; WOW64; Trident/7.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; .NET4.0C; .NET4.0E)", "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.3; WOW64; Trident/7.0; .NET4.0C; .NET4.0E; .NET CLR 2.0.50727; .NET CLR 3.0.30729; .NET CLR 3.5.30729)", "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Win64; x64; Trident/6.0)", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/44.0.2403.157 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36", "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36", "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.109 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.63 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.89 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.65 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36", "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36", "Mozilla/5.0 (Windows NT 6.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36", "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/52.0.2743.116 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/57.0.2987.133 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.99 Safari/537.36", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.104 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.89 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.113 Safari/537.36", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36", "Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36", "Mozilla/5.0 (Windows NT 6.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36", "Mozilla/5.0 (Windows NT 6.2; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36", "Mozilla/5.0 (Windows NT 6.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36", "Mozilla/5.0 (Windows NT 5.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.94 Safari/537.36", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.106 Safari/537.36", "Mozilla/5.0 (Unknown; Linux) AppleWebKit/538.1 (KHTML, like Gecko) Chrome/v1.0.0 Safari/538.1", "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36", "Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.109 Safari/537.36", "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.79 Safari/537.36", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36", ] def random_ip(): a = str(random.randint(1, 255)) b = str(random.randint(1, 255)) c = str(random.randint(1, 255)) d = str(random.randint(1, 255)) ipaddress = '{}.{}.{}.{}'.format(a, b, c, d) return ipaddress def download_mp4(url, mydir): session = HTMLSession() headers = {'Accept-Language': 'zh-CN,zh;q=0.9', 'User-Agent': random.choice(ualist), 'X-Forwarded-For': random_ip(), 'referer': 'http://91porn.com', 'Content-Type': 'multipart/form-data; session_language=cn_CN'} req = session.get(url, headers=headers) filename = os.path.join(mydir, '1.mp4') with open(filename, 'wb') as f: f.write(req.content) def download_img(url, mydir): session = HTMLSession() headers = {'Accept-Language': 'zh-CN,zh;q=0.9', 'User-Agent': random.choice(ualist), 'X-Forwarded-For': random_ip(), 'referer': 'http://91porn.com', 'Content-Type': 'multipart/form-data; session_language=cn_CN'} req = session.get(url, headers=headers) with open(os.path.join(mydir, 'thumb.png'), 'wb') as f: f.write(req.content) def detail_page(url): headers = {'Accept-Language': 'zh-CN,zh;q=0.9', 'User-Agent': random.choice(ualist), 'X-Forwarded-For': random_ip(), 'referer': 'http://91porn.com', 'Content-Type': 'multipart/form-data; session_language=cn_CN'} session = HTMLSession() r = session.get(url, headers=headers) try: mp4url = r.html.search('<source src="{}" type='video/mp4'>')[0] title = r.html.search('<div id="viewvideo-title">{}</div>')[0].strip() imgurl = r.html.search('poster="{}"')[0] if not os.path.exists(title): try: os.makedirs(title) print('开始下载', title) download_img(imgurl, title) download_mp4(mp4url, title) print(title, '视频和缩略图下载完成') except Exception as e: traceback.print_exc() else: print('已存在文件夹{},跳过'.format(title)) except Exception as e: traceback.print_exc() def main(page): for i in range(page + 1): headers = {'Accept-Language': 'zh-CN,zh;q=0.9', 'User-Agent': random.choice(ualist), 'X-Forwarded-For': random_ip(), 'referer': 'http://91porn.com', 'Content-Type': 'multipart/form-data; session_language=cn_CN'} session = HTMLSession() page_url = session.get('http://91porn.com/v.php?next=watch&page={}'.format(i), headers=headers) for url in page_url.html.absolute_links: if '91porn.com/view_video.php' in url: print('即将下载', url) detail_page(url) if __name__ == '__main__': main(4250)#这里是91视频网站的分页数 |
2018.02.28更新：网速测试
|
1 |
!pip install speedtest-cli |
|
1 |
!speedtest-cli |
2018.03.02更新：如何重新启动Google Colab？
|
1 |
!kill -9 -1 |
文章参考：
文章评论
好像不能调整字体大小啊，代码太小了，看起来不舒服，没有在sublime text3上编辑代码舒服。
非常感谢！很有用
这个只能网业打开 那问题来了 怎么保持长久运行呢 必况跑项目测试要很久
我在vps上用ie打开 长久都会掉cookiess 然后就丢失了
@布丁 最长运行是12个小时，超过之后会重新分配，官方已经说明了。
不错，谢谢分享
鸟儿叫，花儿笑，一年一季春来到！
不错不错！内容感觉好极了！