Git Product home page Git Product logo

Comments (29)

DominikDeak avatar DominikDeak commented on May 22, 2024 6

Unfortunately, search performance is becoming a significant bottleneck and often fails in my case. Sample error message:

conan search "boost/*" --remote=all --raw

ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search?q=boost%2F%2A (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

from conan-center-index.

nminkov avatar nminkov commented on May 22, 2024 4

The problem persists. Any solution insight?

conan search -r all openssl
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search?q=openssl (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

from conan-center-index.

memsharded avatar memsharded commented on May 22, 2024 3

Hi all,

yes, we are aware of this limitation. It is in our todo list, but as it is a complex thing that requires changes in the backend it will take some time. Thanks very much for your patience.

from conan-center-index.

uilianries avatar uilianries commented on May 22, 2024 1

Discord? That's cool, thanks for sharing! haha

from conan-center-index.

memsharded avatar memsharded commented on May 22, 2024 1

Moved to conan-center-index, as this is mostly about the ConanCenter, not the client side.

from conan-center-index.

AndWass avatar AndWass commented on May 22, 2024 1

Also hit this issue.

It also happens if I have a range in conanfile.txt:

[requires]
boost/[>=1.73]
$> conan install -if build .
Configuration:
[settings]
arch=x86_64
arch_build=x86_64
build_type=Release
compiler=Visual Studio
compiler.runtime=MD
compiler.version=16
os=Windows
os_build=Windows
[options]
[build_requires]
[env]

ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search?q=boost%2F%2A&ignorecase=False (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

Output of time conan search boost/* -r all:

ERROR: HTTPSConnectionPool(host='api.bintray.com', port=443): Max retries exceeded with url: /conan/conan/conan-center/v1/conans/search?q=boost%2F%2A (Caused by ReadTimeoutError("HTTPSConnectionPool(host='api.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://api.bintray.com/conan/conan/conan-center
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

real    4m2.708s
user    0m0.015s
sys     0m0.000s

from conan-center-index.

tjwrona avatar tjwrona commented on May 22, 2024 1

@uilianries I appreciate the feedback. Unfortunately the problem I am having is that I want to be able to specify version ranges within my conanfiles for different packages I am consuming. Since these dependencies are downloaded during my CMake run with cmake-conan (https://github.com/conan-io/cmake-conan) i don't think there is any way to avoid this problem beacuse CMake will always be using the command line to fetch the dependencies.

For now I will have to just hard-code all of the version numbers to not be ranges, but I would prefer to be able to make my dependencies more flexible than that.

from conan-center-index.

lasote avatar lasote commented on May 22, 2024

Reported to bintray. I don't think that incrementing the default timeout on the client is a sane solution, a list of recipes should be retrieved in milliseconds.

from conan-center-index.

uilianries avatar uilianries commented on May 22, 2024

Thanks!

from conan-center-index.

uilianries avatar uilianries commented on May 22, 2024

UPDATE:

Running Conan 1.17.0:

real 1m47,145s
user 0m0,861s
sys 0m0,139s

from conan-center-index.

Minimonium avatar Minimonium commented on May 22, 2024

Reported by the users again. 😅

from conan-center-index.

uilianries avatar uilianries commented on May 22, 2024

Do you have a reference to the comment? just in case ...

from conan-center-index.

uilianries avatar uilianries commented on May 22, 2024

Anyway, I can confirm such behavior

$ conan --version
Conan version 1.22.2
$ time conan search -r conan-center
DEBUG :conan_api.py   [163]: INIT: Using config '/home/uilian/.conan/conan.conf' [2020-02-21 18:05:31,835]
DEBUG :tracer.py      [156]: CONAN_API: search_recipes(remote_name=conan-center,case_sensitive=False) [2020-02-21 18:05:31,836]
DEBUG :rest_client_common.py[156]: REST: ping: https://conan.bintray.com/v1/ping [2020-02-21 18:05:31,838]
DEBUG :rest_client.py [58]: REST: Cached capabilities for the remote: ['complex_search', 'checksum_deploy', 'revisions'] [2020-02-21 18:05:32,749]
DEBUG :rest_client_common.py[183]: REST: get: https://conan.bintray.com/v2/conans/search [2020-02-21 18:05:32,749]
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v2/conans/search (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

real	4m3,414s
user	0m0,660s
sys	0m0,083s

from conan-center-index.

Minimonium avatar Minimonium commented on May 22, 2024

It was from the #include Discord server.
They were trying to use the

conan search liii* -r=conan-center

And getting timeout errors (the default is 60 seconds after all)

from conan-center-index.

kaihowl avatar kaihowl commented on May 22, 2024

Probably observing the problem again right now. Details: https://twitter.com/kaihowl/status/1267750305443262465

from conan-center-index.

hewigovens avatar hewigovens commented on May 22, 2024

I just installed and tried searching boost, it took 2min50s to get the results, Is someone working on this?

$ time sudo conan search "boost" --remote=conan-center
Existing package recipes:

boost/1.64.0@conan/stable
boost/1.65.1@conan/stable
boost/1.66.0@conan/stable
boost/1.67.0@conan/stable
boost/1.68.0@conan/stable
boost/1.69.0
boost/1.69.0@conan/stable
boost/1.70.0
boost/1.70.0@conan/stable
boost/1.71.0
boost/1.71.0@conan/stable
boost/1.72.0
boost/1.73.0

real	2m50.146s
user	0m0.530s
sys	0m0.066s

BTW: we manage our c++ project internal dependencies (boost/json/gtest/protobuf) in a script, trying to find a usable package manager

from conan-center-index.

uilianries avatar uilianries commented on May 22, 2024

I can say https://conan.io/center is much much faster than the command line when looking for patterns.

from conan-center-index.

kaihowl avatar kaihowl commented on May 22, 2024

@uilianries I observe this timeout during the resolution of version ranges as part of conan create. See https://gitlab.com/ptr-project/catch2_extended/-/jobs/589079001 for example.

 raise ConanConnectionError(("%s\n\nUnable to connect to %s=%s\n" +
 conans.errors.ConanConnectionError: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search?q=boost&ignorecase=False (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

The only way to remedy this seems increasing the timeout, pinning the version, or using lockfiles. I don't have any good logical reason for pinning or lockfiles as far as I can see. Increasing the timeout seems ok. Yet, the performance is surprisingly slow for a simple query. Is this something that should be reported here or is there a better place (or even an existing ticket) for this?

from conan-center-index.

uilianries avatar uilianries commented on May 22, 2024

@kaihowl Bintray reported an internal problem hours ago: https://status.bintray.com/

Please, try again, it should be solved now.

from conan-center-index.

kaihowl avatar kaihowl commented on May 22, 2024
{"_action": "REST_API_CALL", "duration": 117.91095399856567, "headers": {"User-Agent": "Conan/1.25.2 (Python 3.7.7) python-requests/2.22.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": ""}, "met
hod": "GET", "time": 1591800609.898228, "url": "https://conan.bintray.com/v2/conans/search?q=boost&ignorecase=False"}

Still observing the issue. For "smaller" packages like sqlite3 I see the same problem, yet on a smaller scale:

{"_action": "REST_API_CALL", "duration": 20.84811496734619, "headers": {"User-Agent": "Conan/1.25.2 (Python 3.7.7) python-requests/2.22.0", "X-Client-Anonymous-Id": "**********", "X-Client-Id": ""}, "method": "GET", "time": 1591800488.0407581, "url": "https://conan.bintray.com/v2/conans/search?q=sqlite3&ignorecase=False"}

I don't think the reported outage correlates with this problem. I continually observe this problem since my original report, @uilianries.

from conan-center-index.

kuklinistvan avatar kuklinistvan commented on May 22, 2024

I'm experiencing this too :(

from conan-center-index.

thejohnfreeman avatar thejohnfreeman commented on May 22, 2024

4 minutes seems like too long.

$ time conan search "boost/*" -r=conan-center
Existing package recipes:

boost/1.64.0@conan/stable
boost/1.65.1@conan/stable
boost/1.66.0@conan/stable
boost/1.67.0@conan/stable
boost/1.68.0@conan/stable
boost/1.69.0
boost/1.69.0@conan/stable
boost/1.70.0
boost/1.70.0@conan/stable
boost/1.71.0
boost/1.71.0@conan/stable
boost/1.72.0
boost/1.73.0

real    3m53.877s
user    0m0.483s
sys     0m0.060s

from conan-center-index.

sonman avatar sonman commented on May 22, 2024

I am using a conan center on premise. Search takes sometimes minutes because local ConanCenter waits for bintray to answer. The conan search has therefore a not so good user experience.

from conan-center-index.

tjwrona avatar tjwrona commented on May 22, 2024

@AndWass this is also happening for me too. I noticed it when I tried to add a range in a conanfile "requires" statement.

I then tried doing a manual search and it failed. I stumbled upon this ticket and it looks like I am experiencing the same problem as others.

Searching for boost fails:

>conan search -r all boost
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search?q=boost (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

Searching in general fails:

>conan search -r conan-center
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/search (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))

Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

Searching for a specific package version does work (although it is quite slow... took about 10 seconds):

>conan search -r conan-center boost/1.75.0
Existing package recipes:

boost/1.75.0

I am using Conan version 1.29.2.

@memsharded, do you have any updates on the status of this issue? This is causing a big bottleneck in my development workflow.

from conan-center-index.

uilianries avatar uilianries commented on May 22, 2024

@tjwrona1992 Indeed the timeout persists. As alternative, you can use https://conan.io/center. The web search is pretty fast.

For the current timeout problem, we don't have a date for a fix, not soon at least. It requires maintenance on server side, not on Conan client.

from conan-center-index.

blockspacer avatar blockspacer commented on May 22, 2024

Looks like conan search not using bloom filter for popular packages, see https://stackoverflow.com/questions/18447874/google-chrome-usage-of-bloom-filter

from conan-center-index.

aleksa2808 avatar aleksa2808 commented on May 22, 2024

+1

I was planning to experiment with a VS Code extension for finding/installing packages, but after observing conan-center queries being this slow (timing out on most popular packages for me) I had to hang up my hat for now. ☚ī¸

from conan-center-index.

AJIOB avatar AJIOB commented on May 22, 2024

I cannot mirror a specific boost version:

Downloading conanmanifest.txt
Downloading conanfile.py
Downloading conan_export.tgz
Downloading conan_sources.tgz
boost/1.72.0: Getting the complete package list from 'boost/1.72.0#0'...
ERROR: HTTPSConnectionPool(host='conan.bintray.com', port=443): Max retries exceeded with url: /v1/conans/boost/1.72.0/_/_/search (Caused by ReadTimeoutError("HTTPSConnectionPool(host='conan.bintray.com', port=443): Read timed out. (read timeout=60.0)"))
Unable to connect to conan-center=https://conan.bintray.com
1. Make sure the remote is reachable or,
2. Disable it by using conan remote disable,
Then try again.

Do we have any solution?

from conan-center-index.

jgsogo avatar jgsogo commented on May 22, 2024

This is not an issue anymore with latest changes to Artifactory

from conan-center-index.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤ī¸ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.