Comments (6)
I am not sure if you are in the virtualenv.
RE scrapy import error:
You should not get from scrapy import log
import error if you are running it from /Users/josefson/virtualenvs/portia/bin/python
.
RE twistd error:
Its referring to /usr/bin/twistd
but it should point to /Users/josefson/virtualenvs/portia/bin/twistd
.
from portia.
Ok gonna print my pwd, which python/pip and pip list for more info.
Now i am getting this:
/Users/josefson/virtualenvs/portia/bin/twistd: Unknown command: slyd
entire thing:
➜ ~VIRTUAL_ENV pwd
/Users/josefson/virtualenvs/portia
➜ ~VIRTUAL_ENV which python
/Users/josefson/virtualenvs/portia/bin/python
➜ ~VIRTUAL_ENV which pip
/Users/josefson/virtualenvs/portia/bin/pip
➜ ~VIRTUAL_ENV pip list
cffi (0.8.2)
cryptography (0.3)
cssselect (0.9.1)
jsonschema (2.3.0)
loginform (0.9)
lxml (3.3.3)
numpy (1.8.1)
pip (1.5.4)
pycparser (2.10)
pyOpenSSL (0.14)
queuelib (1.1.1)
scrapely (0.10, /Users/josefson/virtualenvs/portia/src/scrapely)
Scrapy (0.22.2)
setuptools (2.2)
six (1.6.1)
slybot (0.9)
Twisted (13.2.0)
w3lib (1.5)
wsgiref (0.1.2)
zope.interface (4.1.1)
➜ ~VIRTUAL_ENV ll
total 0
drwxr-xr-x 36 josefson staff 1.2K Apr 1 16:33 bin
drwxr-xr-x 3 josefson staff 102B Apr 1 16:23 include
drwxr-xr-x 3 josefson staff 102B Apr 1 16:23 lib
drwxr-xr-x 7 josefson staff 238B Apr 1 16:25 portia
drwxr-xr-x 4 josefson staff 136B Apr 1 16:27 src
➜ ~VIRTUAL_ENV twistd -n slyd
Usage: twistd [options]
Options:
--savestats save the Stats object rather than the text output of the
profiler.
-o, --no_save do not save state on shutdown
-e, --encrypted The specified tap/aos file is encrypted.
-n, --nodaemon don't daemonize, don't use default umask of 0077
--originalname Don't try to change the process name
--syslog Log to syslog, not to file
--euid Set only effective user-id rather than real user-id.
(This option has no effect unless the server is running
as root, in which case it means not to shed all
privileges after binding ports, retaining the option to
regain privileges in cases such as spawning processes.
Use with caution.)
-l, --logfile= log to a specified file, - for stdout
--logger= A fully-qualified name to a log observer factory to use
for the initial log observer. Takes precedence over
--logfile and --syslog (when available).
-p, --profile= Run in profile mode, dumping results to specified file
--profiler= Name of the profiler to use (profile, cprofile, hotshot).
[default: hotshot]
-f, --file= read the given .tap file [default: twistd.tap]
-y, --python= read an application from within a Python file (implies
-o)
-s, --source= Read an application from a .tas file (AOT format).
-d, --rundir= Change to a supplied directory before running [default:
.]
--prefix= use the given prefix when syslogging [default: twisted]
--pidfile= Name of the pidfile [default: twistd.pid]
--chroot= Chroot to a supplied directory before running
-u, --uid= The uid to run as.
-g, --gid= The gid to run as.
--umask= The (octal) file creation mask to apply.
--help-reactors Display a list of possibly available reactor names.
--version Print version information and exit.
--spew Print an insanely verbose log of everything that happens.
Useful when debugging freezes or locks in complex code.
-b, --debug Run the application in the Python Debugger (implies
nodaemon), sending SIGUSR2 will drop into debugger
-r, --reactor= Which reactor to use (see --help-reactors for a list of
possibilities)
--help Display this help and exit.
twistd reads a twisted.application.service.Application out of a file and runs
it.
Commands:
conch A Conch SSH service.
dns A domain name server.
ftp An FTP server.
inetd An inetd(8) replacement.
mail An email service
manhole An interactive remote debugger service accessible via
telnet and ssh and providing syntax coloring and basic line
editing functionality.
manhole-old An interactive remote debugger service.
news A news server.
portforward A simple port-forwarder.
procmon A process watchdog / supervisor
socks A SOCKSv4 proxy service.
telnet A simple, telnet-based remote debugging service.
web A general-purpose web server which can serve from a
filesystem or application resource.
words A modern words server
xmpp-router An XMPP Router server
/Users/josefson/virtualenvs/portia/bin/twistd: Unknown command: slyd
from portia.
you should deactivate and then activate your virtualenv
it works for me
from portia.
Same thing
➜ portia pwd
/Users/josefson/virtualenvs/portia
➜ portia source bin/activate
➜ ~VIRTUAL_ENV deactivate
➜ portia source bin/activate
➜ ~VIRTUAL_ENV twistd -n slyd
/Users/josefson/virtualenvs/portia/bin/twistd: Unknown command: slyd
from portia.
The documentation says you need to be in slyd
to run the twisted command.
Your pwd should say: /Users/josefson/virtualenvs/portia/slyd
And then this should work: /Users/josefson/virtualenvs/portia/bin/twistd -n slyd
from portia.
Well... i did it at first but it did not work.
➜ ~WORKON_HOME mkvirtualenv portia
New python executable in portia/bin/python
Installing setuptools, pip...done.
➜ ~WORKON_HOME workon portia
➜ ~WORKON_HOME which pip
/Users/josefson/virtualenvs/portia/bin/pip
➜ ~WORKON_HOME cd portia
➜ ~VIRTUAL_ENV git clone https://github.com/scrapinghub/portia
Cloning into 'portia'...
remote: Counting objects: 3210, done.
remote: Compressing objects: 100% (848/848), done.
remote: Total 3210 (delta 2317), reused 3204 (delta 2312)
Receiving objects: 100% (3210/3210), 1.92 MiB | 339.00 KiB/s, done.
Resolving deltas: 100% (2317/2317), done.
Checking connectivity... done.
➜ ~VIRTUAL_ENV pwd
/Users/josefson/virtualenvs/portia
➜ ~VIRTUAL_ENV cd portia/slyd
➜ slyd git:(master) pip install -r requirements.txt
Successfully installed twisted scrapy loginform lxml jsonschema scrapely slybot zope.interface w3lib queuelib pyOpenSSL cssselect six numpy cryptography cffi pycparser
Cleaning up...
➜ slyd git:(master) pwd
/Users/josefson/virtualenvs/portia/portia/slyd
➜ slyd git:(master) twistd -n slyd
Traceback (most recent call last):
File "/usr/bin/twistd", line 14, in <module>
run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/scripts/twistd.py", line 27, in run
app.run(runApp, ServerOptions)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/application/app.py", line 652, in run
runApp(config)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/scripts/twistd.py", line 23, in runApp
_SomeApplicationRunner(config).run()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/application/app.py", line 386, in run
self.application = self.createOrGetApplication()
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/application/app.py", line 446, in createOrGetApplication
ser = plg.makeService(self.config.subOptions)
File "/Users/josefson/virtualenvs/portia/portia/slyd/slyd/tap.py", line 55, in makeService
root = create_root(config)
File "/Users/josefson/virtualenvs/portia/portia/slyd/slyd/tap.py", line 25, in create_root
from scrapy import log
ImportError: No module named scrapy
But somehow now it is working. So i am closing it. Maybe the deactivate / activate thing did the trick... who knows
from portia.
Related Issues (20)
- how to install it on Mac? HOT 1
- can we used it without the fucking Docker?
- Unexpected Error
- can we use it on Mac OS without the mother fuck docker?
- Could not determine app directory
- see nothing on http://localhost:9001
- Portia crashes on Single Page Applications
- Failed loading page HOT 1
- Issue on creating project
- Set download delay HOT 1
- https://boot2docker.io/ HOT 1
- docker compose up build failed
- standard_init_linux.go:219: exec user process caused: exec format error
- Blank white page on localhost and portia says: PermisionDenied HOT 1
- 'what would you like to scrape' cannot enter url HOT 2
- Browser version error message HOT 4
- 项目没有人维护了吗 HOT 1
- is anyone able to run this? HOT 3
- How to get fields exported in correct order?
- Need Help scraping API HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from portia.