Add support for handling dynamic manifest creation about robottelo HOT 30 CLOSED

You could update requirements.txt to include:

paramiko
selenium
sauceclient
python-bugzilla
git+https://github.com/RedHatQE/python-stageportal.git

from robottelo.

hmmm it also requires 2 more deps:

Requires:        python-requests python-rhsm

For python-requests we can add requests to requirements.txt but I could not find python-rhsm in Pypi.

from robottelo.

Just filed RedHatQE/python-stageportal#1 to help us install python-stageportal through pip.

from robottelo.

yeah, making it a part of library is challenging,
neither a setup.py, nor the rpm exists.

Having a pip install would be cool.

In the meantime I plan to explore the above tool with it's options.

from robottelo.

Looks like we already have a setup.py, but it needs to get merged.

from robottelo.

Yup. @elyezer got his PR merged RedHatQE/python-stageportal#2 💯

from robottelo.

ME tried install and up until now had to perform the below steps, please let me know if I am missing something.

1. pip install -e git+https://github.com/RedHatQE/python-stageportal.git#egg=stageportal
2. It installs , but upon running command fails for pip install -e git+https://github.com/RedHatQE/python-stageportal.git
3. Installed openssl-devel package and ran 2) installs fine
4. running commands fails as it requires M2Crypto module and upon installing it, Fails for swig
5. Installed swig
6. Fails for /usr/include/openssl/opensslconf.h:31: Error: CPP #error ""This openssl-devel package does not work your architecture?"". Use the -cpperraswarn option to continue swig processing.
7. found sol at http://stackoverflow.com/questions/7772965/m2crypto-doesnt-install-in-venv-or-swig-doesnt-define-x86-64-which-breaks
8. trying

from robottelo.

These are python-rhsm dependencies. The python-rhsm setup.py do not specify those dependencies but the spec file does.

I could not get M2Crypto installed in a virtualenv too. I'll send a feedback if I get something here.

from robottelo.

@elyezer thanks!

from robottelo.

ah! lucky, I found the way out.

9. with pip install m2crypto --no-clean (no clean required for the next solution)

10. cd /venv/build/M2Crypto
    chmod u+x fedora_setup.sh
    ./fedora_setup.sh build
    ./fedora_setup.sh install

11. Fails for iniparse, pip install iniparse
    Looks like there are too many deps for pythom-rhsm, and also could be that I am using virtual-envs

12. Now fails for rpm module

from robottelo.

Here are all the requirements: m2crypto (install using the @kbidarkar steps), python-iniparse (pip install iniparse), rpm-python (pip install -e git+git://rpm.org/rpm-python.git)

I have tried to install rpm-python on my mac but it fails because could not find RPM, I think that it will install on Fedora (will try that later).

from robottelo.

@elyezer thanks

I tried with the above command for rpm-python, but it fails with the below output.

@omaciel : From tomorrow, Me thinking of abandoning virtual-env and happily use fedora python packages :)
Installing from pip is getting too complicated.

gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python2.7 -c src/rpmte-py.c -o build/temp.linux-x86_64-2.7/src/rpmte-py.o

src/rpmte-py.c: In function ‘rpmte_AddedKey’:

src/rpmte-py.c:124:5: warning: implicit declaration of function ‘rpmteAddedKey’ [-Wimplicit-function-declaration]

src/rpmte-py.c: In function ‘rpmte_DependsOnKey’:

src/rpmte-py.c:130:5: warning: implicit declaration of function ‘rpmteDependsOnKey’ [-Wimplicit-function-declaration]

src/rpmte-py.c: In function ‘rpmte_DS’:

src/rpmte-py.c:172:5: error: too many arguments to function ‘rpmdsLink’

In file included from src/rpmds-py.h:6:0,

             from src/rpmte-py.c:6:

/usr/include/rpm/rpmds.h:93:7: note: declared here

src/rpmte-py.c: In function ‘rpmte_FI’:

src/rpmte-py.c:182:5: error: too many arguments to function ‘rpmfiLink’

In file included from src/rpmfi-py.h:6:0,

             from src/rpmte-py.c:7:

/usr/include/rpm/rpmfi.h:105:7: note: declared here

error: command 'gcc' failed with exit status 1

Command /home/kedar/envs/robotello/bin/python -c "import setuptools; file='/home/kedar/envs/robotello/src/rpmng/setup.py'; exec(compile(open(file).read().replace('\r\n', '\n'), file, 'exec'))" develop --no-deps failed with error code 1 in /home/kedar/envs/robotello/src/rpmng
Storing complete log in /home/kedar/.pip/pip.log

from robottelo.

@blrm : What do you think?

from robottelo.

Also what I have realised is that error reporting is not that great, when something fails.

While installing rpm module via pip, if it fails saying something : **pkgconfig('rpm') , make sure you have rpm-devel package installed on the machine.

from robottelo.

@kbidarkar I am failing on the same error you posted. It could be that there is another dependency I am missing, but it isn't clear from the traceback. I'll keep investigating.

from robottelo.

Looks like the best way forward is it use --system-site-packages options while creating the VIRTUAL-ENV

virtualenv --system-site-packages robottelo

and once created in the above fashion, all the troublesome module could be fetched easily from the system-site-packages, from the distribution packages.

Also to still isolate the modules, we can go ahead installing a module for the local virtual-env using the below command.

(what is -I option, pip install --help)

Example:-
pip install -I boto

I think to simplify things that would be the way ahead, atleast for modules like rpm-python.

from robottelo.

Also I installed boto via the above method and checked the version, the version is from the local virtual-env .

So it's obvious that upon installing pip modules inside the virtual-env, it first looks for the local module and then for the system/global module if unavailable.

from robottelo.

Using system-site-packages will speed up things because the dependent packages was not designed to be installed using pip.

Would be better to have an isolated environment but I think that we should not lose time making all the packages able to be installed in a virtualenv.

from robottelo.

I completely agree with @elyezer.
I followed the above said procedure and me currently able to work with stageportal.

from robottelo.

@kbidarkar thank you to share the required steps

What about creating a script that helps setup the required environment (using requirements.txt and installing python-* packages)?

from robottelo.

yeah, I plan to do so. Just created a user, But just stuck with moving forward. looks like something else too is required.

from robottelo.

One more thing we also require the ca certs from python-rhsm, strange installing from pip doesn't populate them.

from robottelo.

vkuznets helped and provided me all the details/links related to adding SKU's to a new customer, the missing ca's and getting started.

Look's like he is able to add sku's to a new customer created, but me unable to, may be because of some missing deps.

from robottelo.

The ca cert interface seems to be provided by https://github.com/candlepin/python-rhsm/blob/master/src/certificate.c, and the certificates installation is accomplished by install section on the spec file (look at last line):

%install
rm -rf %{buildroot}
%{__python} setup.py install -O1 --skip-build --root %{buildroot}
mkdir -p %{buildroot}%{_sysconfdir}/rhsm/ca
install etc-conf/ca/*.pem %{buildroot}%{_sysconfdir}/rhsm/ca

I'm not sure if we can get the same result when installing with pip.

from robottelo.

@elyezer , thanks for the above info, good to know.

Me stuck with this trace-back, need some help with this. some clue or pointers would help.

https://www.refheap.com/21268

from robottelo.

@kbidarkar do you have the exception line? I could not figure something out from the traceback only.

from robottelo.

nope, doesn't throw any exception, just the bound method line on trying the below command.

--login xx --password xxxx --candlepin $CANDLEPIN --portal $PORTAL --api $API --action sku_add --sku-id SKUxxxx --sku-quantity 5 --sku-start-date 2013-11-26 --verbose

2013-11-26 23:47:06,846 ERROR <bound method StagePortal._get_subscriptions of <main.StagePortal object at 0x2917f90>> failed after 40 tries, last result: set(['3059464', '3059465', '3059466', '3059467', '3059461', '3059471', '3059470', '3059468', '3059469'])
2013-11-26 23:47:06,846 ERROR Can't find subscriptions
None

from robottelo.

Now It's working, looks like there were some disk space issues earlier on the back end according to vkuznets yesterday. me moving ahead, with the actual task.

from robottelo.

Initial functionality in https://github.com/omaciel/robottelo/pull/111

from robottelo.

Done.

from robottelo.