Python 官方文档:入门教程 => 点击学习
这篇文章主要介绍了python爬虫如何采集微博视频数据,具有一定借鉴价值,感兴趣的朋友可以参考下,希望大家阅读完这篇文章之后大有收获,下面让小编带着大家一起了解一下。知识点requestspprint开发环境版 本:python 3.8-编
这篇文章主要介绍了python爬虫如何采集微博视频数据,具有一定借鉴价值,感兴趣的朋友可以参考下,希望大家阅读完这篇文章之后大有收获,下面让小编带着大家一起了解一下。
requests
pprint
版 本:python 3.8
-编辑器:PyCharm 2021.2
作用:批量获取互联网数据(文本, 图片, 音频, 视频)
本质:一次次的请求与响应
导入所需模块
import requestsimport pprint
找到目标网址
打开开发者工具,选中Fetch/XHR,选中数据所在的标签,找到目标所在url
https://www.weibo.com/tv/api/component?page=/tv/channel/4379160563414111/editor
发送网络请求
headers = { 'cookie': '', 'referer': 'Https://weibo.com/tv/channel/4379160563414111/editor', 'user-agent': '',}data = { 'data': '{"Component_Channel_Editor":{"cid":"4379160563414111","count":9}}'}url = 'https://www.weibo.com/tv/api/component?page=/tv/channel/4379160563414111/editor'JSON_data = requests.post(url=url, headers=headers, data=data).json()
获取数据
json_data_2 = requests.post(url=url_1, headers=headers, data=data_1).json()
筛选数据
dict_urls = json_data_2['data']['Component_Play_Playinfo']['urls']video_url = "https:" + dict_urls[list(dict_urls.keys())[0]]print(title + "\t" + video_url)
保存数据
video_data = requests.get(video_url).contentwith open(f'video\\{title}.mp4', mode='wb') as f: f.write(video_data)print(title, "爬取成功................")
完整代码
import requestsimport pprintheaders = { 'cookie': '添加自己的', 'referer': 'https://weibo.com/tv/channel/4379160563414111/editor', 'user-agent': '',}data = { 'data': '{"Component_Channel_Editor":{"cid":"4379160563414111","count":9}}'}url = 'https://www.weibo.com/tv/api/component?page=/tv/channel/4379160563414111/editor'json_data = requests.post(url=url, headers=headers, data=data).json()print(json_data)ccs_list = json_data['data']['Component_Channel_Editor']['list']next_cursor = json_data['data']['Component_Channel_Editor']['next_cursor']for ccs in ccs_list: oid = ccs['oid'] title = ccs['title'] data_1 = { 'data': '{"Component_Play_Playinfo":{"oid":"' + oid + '"}}' } url_1 = 'https://weibo.com/tv/api/component?page=/tv/show/' + oid json_data_2 = requests.post(url=url_1, headers=headers, data=data_1).json() dict_urls = json_data_2['data']['Component_Play_Playinfo']['urls'] video_url = "https:" + dict_urls[list(dict_urls.keys())[0]] print(title + "\t" + video_url) video_data = requests.get(video_url).content with open(f'video\\{title}.mp4', mode='wb') as f: f.write(video_data) print(title, "爬取成功................")
感谢你能够认真阅读完这篇文章,希望小编分享的“Python爬虫如何采集微博视频数据”这篇文章对大家有帮助,同时也希望大家多多支持编程网,关注编程网Python频道,更多相关知识等着你来学习!
--结束END--
本文标题: Python爬虫如何采集微博视频数据
本文链接: https://www.lsjlt.com/news/301220.html(转载时请注明来源链接)
有问题或投稿请发送至: 邮箱/279061341@qq.com QQ/279061341
下载Word文档到电脑,方便收藏和打印~
2024-03-01
2024-03-01
2024-03-01
2024-02-29
2024-02-29
2024-02-29
2024-02-29
2024-02-29
2024-02-29
2024-02-29
回答
回答
回答
回答
回答
回答
回答
回答
回答
回答
0