是否有Amazon.com API来检索产品评论?


76

是否有任何AWS API /服务提供对Amazon出售商品的产品评论的访问权限?我有兴趣通过(ASIN,user_id)元组查找评论。我可以看到,产品广告API将URL返回到包含URL的页面(用于嵌入到IFRAME中),但我对评论数据的机器可读格式感兴趣,如果可能的话。


嗨,Dcrosta:嗯,有一些插件可以做到这一点:例如。对于wordpress,您可以使用ScrapeAZon。我想它确实可以满足您的需求。-试试看

不知道任何API,但这可能会有所帮助。jmcauley.ucsd.edu/data/amazon
Rajarshee Mitra


截至2020年,对此有何更新?我阅读了文档,但在API中找不到更多的灵活性。
mdabdullah

Answers:


36

更新2:

请参阅@jpillora的评论。这可能与Update 1最相关。

我刚刚试用了Product Advertising API(截至2014-09-17),看来该API仅返回指向仅包含评论的iframe的URL。我猜您将不得不刮擦屏幕-尽管我认为那会破坏亚马逊的服务条款。

更新1:

也许。我在下面写了原始答案。我现在没有时间研究这个问题,因为我不再参与与亚马逊评论相关的项目,但是他们在Product Advertising API上的网页上指出:“ Product Advertising API可帮助您使用产品搜索和查找来宣传Amazon产品功能,产品信息和功能,例如“客户评论...”,截至2011年12月8日。因此,我希望有人对此进行调查并发回此处;随时编辑此答案。

原版的:

不。

这是一个有关该事实的有趣的论坛讨论,包括有关其原因的理论:http : //forums.digitalpoint.com/showthread.php?t=1932326

如果我错了,请发表您的发现。我有兴趣获取评论内容,并在可能的情况下允许将评论提交给亚马逊。

您可能需要检查以下链接:http : //reviewazon.com/。我只是偶然发现了它,还没有研究它,但令我感到惊讶的是,我在他们的网站上没有看到有关从Amazon Products Advertising API中删除评论的更新的任何消息,该更新发布在:https:// affiliate -program.amazon.com/gp/advertising/api/detail/main.html


7
我刚刚试用了Product Advertising API(截至2014-09-17),看来该API仅返回指向仅包含评论的iframe的URL。我猜您将不得不刮擦屏幕-尽管我认为那会破坏亚马逊的服务条款。
jpillora

截至15年7月24日,@ jpillora的评论仍然正确。
Cory Dee 2015年

14

这是我的简要介绍-您可以轻松地通过一些工作来检索评论:

countries=['com','co.uk','ca','de']
books=[
        '''http://www.amazon.%s/Glass-House-Climate-Millennium-ebook/dp/B005U3U69C''',
        '''http://www.amazon.%s/The-Japanese-Observer-ebook/dp/B0078FMYD6''',
        '''http://www.amazon.%s/Falling-Through-Water-ebook/dp/B009VJ1622''',
      ]
import urllib2;
for book in books:
    print '-'*40
    print book.split('%s/')[1]
    for country in countries:
        asin=book.split('/')[-1]; title=book.split('/')[3]
        url='''http://www.amazon.%s/product-reviews/%s'''%(country,asin)
        try: f = urllib2.urlopen(url)
        except: page=""
        page=f.read().lower(); print '%s=%s'%(country, page.count('member-review'))
print '-'*40

9
我要指出的是,最近的法律影响(在这里有craiglist的想法)意味着,如果您未经他们的许可而这样做,并且他们打算封锁像您这样的人,那么您可能会使用此脚本来触犯法律
Sammaye 2013年

7

根据亚马逊产品广告API许可协议(https://affiliate-program.amazon.com/gp/advertising/api/detail/agreement.html),具体来说是要点4.b.iii:

您将仅使用产品广告内容...将最终用户吸引到Amazon Site并在Amazon Site上促进销售。

这意味着禁止您显示通过其API购买的亚马逊产品评论在您的网站上销售产品。仅允许将您的网站访问者重定向到亚马逊并获得会员佣金。


4

我会使用类似@mfs的答案。不幸的是,他/她的答案最多只能使用10条评论,因为这是可以在一页上显示的最大值。

您可以考虑以下代码:

import requests

nreviews_re = {'com': re.compile('\d[\d,]+(?= customer review)'), 
               'co.uk':re.compile('\d[\d,]+(?= customer review)'),
               'de': re.compile('\d[\d\.]+(?= Kundenrezens\w\w)')}
no_reviews_re = {'com': re.compile('no customer reviews'), 
                 'co.uk':re.compile('no customer reviews'),
                 'de': re.compile('Noch keine Kundenrezensionen')}

def get_number_of_reviews(asin, country='com'):                                 
    url = 'http://www.amazon.{country}/product-reviews/{asin}'.format(country=country, asin=asin)
    html = requests.get(url).text
    try:
        return int(re.compile('\D').sub('',nreviews_re[country].search(html).group(0)))
    except:
        if no_reviews_re[country].search(html):
            return 0
        else:
            return None  # to distinguish from 0, and handle more cases if necessary

使用1433524767(对于三个感兴趣的国家/地区,评论的数量明显不同)运行它,我得到:

>> print get_number_of_reviews('1433524767', 'com')
3185
>> print get_number_of_reviews('1433524767', 'co.uk')
378
>> print get_number_of_reviews('1433524767', 'de')
16

希望能帮助到你


3

如上文其他人所述,亚马逊已停止在其api中提供评论。但是,我发现这个不错的教程可以对python进行同样的操作。这是他提供的代码,对我有用!他使用python 2.7

#!/usr/bin/env python
# -*- coding: utf-8 -*-
# Written as part of https://www.scrapehero.com/how-to-scrape-amazon-product-reviews-using-python/      
from lxml import html  
import json
import requests
import json,re
from dateutil import parser as dateparser
from time import sleep

def ParseReviews(asin):
    #This script has only been tested with Amazon.com
    amazon_url  = 'http://www.amazon.com/dp/'+asin
    # Add some recent user agent to prevent amazon from blocking the request 
    # Find some chrome user agent strings  here https://udger.com/resources/ua-list/browser-detail?browser=Chrome
    headers = {'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.90 Safari/537.36'}
    page = requests.get(amazon_url,headers = headers).text

    parser = html.fromstring(page)
    XPATH_AGGREGATE = '//span[@id="acrCustomerReviewText"]'
    XPATH_REVIEW_SECTION = '//div[@id="revMHRL"]/div'
    XPATH_AGGREGATE_RATING = '//table[@id="histogramTable"]//tr'
    XPATH_PRODUCT_NAME = '//h1//span[@id="productTitle"]//text()'
    XPATH_PRODUCT_PRICE  = '//span[@id="priceblock_ourprice"]/text()'

    raw_product_price = parser.xpath(XPATH_PRODUCT_PRICE)
    product_price = ''.join(raw_product_price).replace(',','')

    raw_product_name = parser.xpath(XPATH_PRODUCT_NAME)
    product_name = ''.join(raw_product_name).strip()
    total_ratings  = parser.xpath(XPATH_AGGREGATE_RATING)
    reviews = parser.xpath(XPATH_REVIEW_SECTION)

    ratings_dict = {}
    reviews_list = []

    #grabing the rating  section in product page
    for ratings in total_ratings:
        extracted_rating = ratings.xpath('./td//a//text()')
        if extracted_rating:
            rating_key = extracted_rating[0] 
            raw_raing_value = extracted_rating[1]
            rating_value = raw_raing_value
            if rating_key:
                ratings_dict.update({rating_key:rating_value})

    #Parsing individual reviews
    for review in reviews:
        XPATH_RATING  ='./div//div//i//text()'
        XPATH_REVIEW_HEADER = './div//div//span[contains(@class,"text-bold")]//text()'
        XPATH_REVIEW_POSTED_DATE = './/a[contains(@href,"/profile/")]/parent::span/following-sibling::span/text()'
        XPATH_REVIEW_TEXT_1 = './/div//span[@class="MHRHead"]//text()'
        XPATH_REVIEW_TEXT_2 = './/div//span[@data-action="columnbalancing-showfullreview"]/@data-columnbalancing-showfullreview'
        XPATH_REVIEW_COMMENTS = './/a[contains(@class,"commentStripe")]/text()'
        XPATH_AUTHOR  = './/a[contains(@href,"/profile/")]/parent::span//text()'
        XPATH_REVIEW_TEXT_3  = './/div[contains(@id,"dpReviews")]/div/text()'
        raw_review_author = review.xpath(XPATH_AUTHOR)
        raw_review_rating = review.xpath(XPATH_RATING)
        raw_review_header = review.xpath(XPATH_REVIEW_HEADER)
        raw_review_posted_date = review.xpath(XPATH_REVIEW_POSTED_DATE)
        raw_review_text1 = review.xpath(XPATH_REVIEW_TEXT_1)
        raw_review_text2 = review.xpath(XPATH_REVIEW_TEXT_2)
        raw_review_text3 = review.xpath(XPATH_REVIEW_TEXT_3)

        author = ' '.join(' '.join(raw_review_author).split()).strip('By')

        #cleaning data
        review_rating = ''.join(raw_review_rating).replace('out of 5 stars','')
        review_header = ' '.join(' '.join(raw_review_header).split())
        review_posted_date = dateparser.parse(''.join(raw_review_posted_date)).strftime('%d %b %Y')
        review_text = ' '.join(' '.join(raw_review_text1).split())

        #grabbing hidden comments if present
        if raw_review_text2:
            json_loaded_review_data = json.loads(raw_review_text2[0])
            json_loaded_review_data_text = json_loaded_review_data['rest']
            cleaned_json_loaded_review_data_text = re.sub('<.*?>','',json_loaded_review_data_text)
            full_review_text = review_text+cleaned_json_loaded_review_data_text
        else:
            full_review_text = review_text
        if not raw_review_text1:
            full_review_text = ' '.join(' '.join(raw_review_text3).split())

        raw_review_comments = review.xpath(XPATH_REVIEW_COMMENTS)
        review_comments = ''.join(raw_review_comments)
        review_comments = re.sub('[A-Za-z]','',review_comments).strip()
        review_dict = {
                            'review_comment_count':review_comments,
                            'review_text':full_review_text,
                            'review_posted_date':review_posted_date,
                            'review_header':review_header,
                            'review_rating':review_rating,
                            'review_author':author

                        }
        reviews_list.append(review_dict)

    data = {
                'ratings':ratings_dict,
                'reviews':reviews_list,
                'url':amazon_url,
                'price':product_price,
                'name':product_name
            }
    return data


def ReadAsin():
    #Add your own ASINs here 
    AsinList = ['B01ETPUQ6E','B017HW9DEW']
    extracted_data = []
    for asin in AsinList:
        print "Downloading and processing page http://www.amazon.com/dp/"+asin
        extracted_data.append(ParseReviews(asin))
        sleep(5)
    f=open('data.json','w')
    json.dump(extracted_data,f,indent=4)

if __name__ == '__main__':
    ReadAsin()

这是他的网站评论python 2.7抓取的链接




-1

您可以使用Amazon Product Advertising API。它具有响应组“审阅”,您可以将其与操作“ ItemLookup”一起使用。您需要知道ASIN,即产品的唯一商品ID。

设置所有参数并执行签名的URL后,您将收到一个XML,其中包含指向“ IFrameURL”标签下的客户评论的链接。

使用此URL并在从该URL返回的html中使用模式搜索来提取评论。对于html中的每个评论,都会有一个唯一的评论ID,您可以在该ID下获取该特定评论的所有数据。


1
好吧,这正是他在说什么。获得评论的唯一方法是通过iframe,这非常不方便。
user1029829
By using our site, you acknowledge that you have read and understand our Cookie Policy and Privacy Policy.
Licensed under cc by-sa 3.0 with attribution required.