Git Product home page Git Product logo

Comments (6)

utsengar avatar utsengar commented on July 17, 2024

I am not sure if you are in the virtualenv.

RE scrapy import error:
You should not get from scrapy import log import error if you are running it from /Users/josefson/virtualenvs/portia/bin/python.

RE twistd error:
Its referring to /usr/bin/twistd but it should point to /Users/josefson/virtualenvs/portia/bin/twistd.

from portia.

josefson avatar josefson commented on July 17, 2024

Ok gonna print my pwd, which python/pip and pip list for more info.
Now i am getting this:

/Users/josefson/virtualenvs/portia/bin/twistd: Unknown command: slyd

entire thing:

➜  ~VIRTUAL_ENV  pwd
/Users/josefson/virtualenvs/portia
➜  ~VIRTUAL_ENV  which python
/Users/josefson/virtualenvs/portia/bin/python
➜  ~VIRTUAL_ENV  which pip
/Users/josefson/virtualenvs/portia/bin/pip
➜  ~VIRTUAL_ENV  pip list
cffi (0.8.2)
cryptography (0.3)
cssselect (0.9.1)
jsonschema (2.3.0)
loginform (0.9)
lxml (3.3.3)
numpy (1.8.1)
pip (1.5.4)
pycparser (2.10)
pyOpenSSL (0.14)
queuelib (1.1.1)
scrapely (0.10, /Users/josefson/virtualenvs/portia/src/scrapely)
Scrapy (0.22.2)
setuptools (2.2)
six (1.6.1)
slybot (0.9)
Twisted (13.2.0)
w3lib (1.5)
wsgiref (0.1.2)
zope.interface (4.1.1)
➜  ~VIRTUAL_ENV  ll
total 0
drwxr-xr-x  36 josefson  staff   1.2K Apr  1 16:33 bin
drwxr-xr-x   3 josefson  staff   102B Apr  1 16:23 include
drwxr-xr-x   3 josefson  staff   102B Apr  1 16:23 lib
drwxr-xr-x   7 josefson  staff   238B Apr  1 16:25 portia
drwxr-xr-x   4 josefson  staff   136B Apr  1 16:27 src
➜  ~VIRTUAL_ENV  twistd -n slyd
Usage: twistd [options]
Options:
      --savestats      save the Stats object rather than the text output of the
                       profiler.
  -o, --no_save        do not save state on shutdown
  -e, --encrypted      The specified tap/aos file is encrypted.
  -n, --nodaemon       don't daemonize, don't use default umask of 0077
      --originalname   Don't try to change the process name
      --syslog         Log to syslog, not to file
      --euid           Set only effective user-id rather than real user-id.
                       (This option has no effect unless the server is running
                       as root, in which case it means not to shed all
                       privileges after binding ports, retaining the option to
                       regain privileges in cases such as spawning processes.
                       Use with caution.)
  -l, --logfile=       log to a specified file, - for stdout
      --logger=        A fully-qualified name to a log observer factory to use
                       for the initial log observer. Takes precedence over
                       --logfile and --syslog (when available).
  -p, --profile=       Run in profile mode, dumping results to specified file
      --profiler=      Name of the profiler to use (profile, cprofile, hotshot).
                       [default: hotshot]
  -f, --file=          read the given .tap file [default: twistd.tap]
  -y, --python=        read an application from within a Python file (implies
                       -o)
  -s, --source=        Read an application from a .tas file (AOT format).
  -d, --rundir=        Change to a supplied directory before running [default:
                       .]
      --prefix=        use the given prefix when syslogging [default: twisted]
      --pidfile=       Name of the pidfile [default: twistd.pid]
      --chroot=        Chroot to a supplied directory before running
  -u, --uid=           The uid to run as.
  -g, --gid=           The gid to run as.
      --umask=         The (octal) file creation mask to apply.
      --help-reactors  Display a list of possibly available reactor names.
      --version        Print version information and exit.
      --spew           Print an insanely verbose log of everything that happens.
                       Useful when debugging freezes or locks in complex code.
  -b, --debug          Run the application in the Python Debugger (implies
                       nodaemon), sending SIGUSR2 will drop into debugger
  -r, --reactor=       Which reactor to use (see --help-reactors for a list of
                       possibilities)
      --help           Display this help and exit.

twistd reads a twisted.application.service.Application out of a file and runs
it.
Commands:
    conch            A Conch SSH service.
    dns              A domain name server.
    ftp              An FTP server.
    inetd            An inetd(8) replacement.
    mail             An email service
    manhole          An interactive remote debugger service accessible via
                     telnet and ssh and providing syntax coloring and basic line
                     editing functionality.
    manhole-old      An interactive remote debugger service.
    news             A news server.
    portforward      A simple port-forwarder.
    procmon          A process watchdog / supervisor
    socks            A SOCKSv4 proxy service.
    telnet           A simple, telnet-based remote debugging service.
    web              A general-purpose web server which can serve from a
                     filesystem or application resource.
    words            A modern words server
    xmpp-router      An XMPP Router server

/Users/josefson/virtualenvs/portia/bin/twistd: Unknown command: slyd

from portia.

wadecong avatar wadecong commented on July 17, 2024

you should deactivate and then activate your virtualenv
it works for me

from portia.

josefson avatar josefson commented on July 17, 2024

Same thing

➜  portia  pwd
/Users/josefson/virtualenvs/portia
➜  portia  source bin/activate
➜  ~VIRTUAL_ENV  deactivate
➜  portia  source bin/activate
➜  ~VIRTUAL_ENV  twistd -n slyd
/Users/josefson/virtualenvs/portia/bin/twistd: Unknown command: slyd

from portia.

utsengar avatar utsengar commented on July 17, 2024

The documentation says you need to be in slyd to run the twisted command.

Your pwd should say: /Users/josefson/virtualenvs/portia/slyd

And then this should work: /Users/josefson/virtualenvs/portia/bin/twistd -n slyd

from portia.

josefson avatar josefson commented on July 17, 2024

Well... i did it at first but it did not work.

➜  ~WORKON_HOME  mkvirtualenv portia
New python executable in portia/bin/python
Installing setuptools, pip...done.
➜  ~WORKON_HOME  workon portia
➜  ~WORKON_HOME  which pip
/Users/josefson/virtualenvs/portia/bin/pip
➜  ~WORKON_HOME  cd portia
➜  ~VIRTUAL_ENV  git clone https://github.com/scrapinghub/portia
Cloning into 'portia'...
remote: Counting objects: 3210, done.
remote: Compressing objects: 100% (848/848), done.
remote: Total 3210 (delta 2317), reused 3204 (delta 2312)
Receiving objects: 100% (3210/3210), 1.92 MiB | 339.00 KiB/s, done.
Resolving deltas: 100% (2317/2317), done.
Checking connectivity... done.
➜  ~VIRTUAL_ENV  pwd
/Users/josefson/virtualenvs/portia
➜  ~VIRTUAL_ENV  cd portia/slyd
➜  slyd git:(master) pip install -r requirements.txt
Successfully installed twisted scrapy loginform lxml jsonschema scrapely slybot zope.interface w3lib queuelib pyOpenSSL cssselect six numpy cryptography cffi pycparser
Cleaning up...
➜  slyd git:(master) pwd
/Users/josefson/virtualenvs/portia/portia/slyd
➜  slyd git:(master) twistd -n slyd
Traceback (most recent call last):
  File "/usr/bin/twistd", line 14, in <module>
    run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/scripts/twistd.py", line 27, in run
    app.run(runApp, ServerOptions)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/application/app.py", line 652, in run
    runApp(config)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/scripts/twistd.py", line 23, in runApp
    _SomeApplicationRunner(config).run()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/application/app.py", line 386, in run
    self.application = self.createOrGetApplication()
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/twisted/application/app.py", line 446, in createOrGetApplication
    ser = plg.makeService(self.config.subOptions)
  File "/Users/josefson/virtualenvs/portia/portia/slyd/slyd/tap.py", line 55, in makeService
    root = create_root(config)
  File "/Users/josefson/virtualenvs/portia/portia/slyd/slyd/tap.py", line 25, in create_root
    from scrapy import log
ImportError: No module named scrapy

But somehow now it is working. So i am closing it. Maybe the deactivate / activate thing did the trick... who knows

from portia.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.