Python Developer

If you look here I could not get two different spiders to automatically add the results to a mysql… Read more Spider Not Returning All Results After Changing My Item Pipelines To An If And Elif Statement

This is how my spider is set up class CustomSpider(CrawlSpider): name = 'custombot' … Read more Scrapy: What's The Correct Way To Use Start_requests()?

I am following this guide for scraping data from instagram: http://www.spataru.at/scraping-instagr… Read more From Scraper_user.items Import Useritem Importerror: No Module Named Scraper_user.items

I'm trying to scrape the current currency exchange rates. Please could you take a look and see … Read more Scrapy And Python. Nameerror

I'm trying to use this code to get list of urls in sitemap. when i run this, i see no results i… Read more How To Write Python Scrapy Code For Extracting Url's Present In Sitemap Of A Site

I'm trying to crawl this website: http://www.aido.com/eshop/cl_2-c_189-p_185/stationery/pens.ht… Read more Pagination Using Scrapy

I am new to Python and am currently trying to figure out how to scrape data from this web: https://… Read more Scraping Data From A Dynamic Web Database With Python

I'm trying to write a very simple website crawler to list URLs along with referrer and status c… Read more Can't Get Scrapy To Parse And Follow 301, 302 Redirects

I just started using scrapy-splash to retrieve the number of bookings from opentable.com. The follo… Read more Scrapy-splash Active Content Selector Works In Shell But Not With Spider

I am learning scrapy and am having a hard time trying to figure out this issue. My spider will not … Read more Scrapy Twisted Connectionlost Error

Scrapy-redis framework, redis stored xxx: requests have been crawled finished, but the program is s… Read more The Scrapy-redis Program Does Not Close Automatically

I'm facing a problem that when I click on a button, then Javascript handle the action then it r… Read more Scrapy/splash Click On A Button Then Get Content From New Page In New Window

The Scrapy Splash I am using is working just fine on my local machine, but it returns this error wh… Read more Scrapy Splash On Ubuntu Server: Got An Unexpected Keyword Argument 'encoding'

I'm using scrapy to build a spider to monitor prices on a website. The website isn't consis… Read more If Statement Within Scrapy Item Declaration

So I'm building a scraper that imports a .csv excel file which has one row of ~2,400 websites (… Read more Scrapy - Importing Excel .csv As Start_url

I seem to be missing something very simple. All i want to do is use ; as a delimiter in the CSV exp… Read more Modifiying Csv Export In Scrapy

I'm getting error installing Scrapy on my ubuntu box. I'm using pip to install Scrapy. I… Read more Installing Scrapy On Ubuntu 14.04 Fails

this is my xml file i parse it like this: self.doc=etree.parse(xmlFile) masterI Solution 1: In X… Read more Parse Xpath From Xml File Should Contain '

I want to crawl a website, there are more than 10 pages every page has 10 links, the spider will g… Read more Scrapy And Selenium : Only Scrape Two Pages

I try to instantiate multiple spiders. The first one works fine, but the second one gives me an err… Read more Running More Than One Spider In A For Loop