crawlab-core

Backend core modules for Crawlab

crawlab-core's People

Contributors

Stargazers

Watchers

crawlab-core's Issues

python dependencies parsing error

e.g. expect 'crawlab-sdk' but got 'crawlab', and dependency fails to install.

the regex expression in function _getDependenciesRequirementsTxt from env_deps.go cannot cover some special char like '-'.

	// regex pattern
	pattern, err := regexp.Compile("(\\w+)([=<>]=.*)?")
	if err != nil {
		return nil, trace.TraceError(err)
	}

net/http: HTTP/1.x transport connection broken: http: ContentLength=184 with Body length 238

版本：0.6.0+
场景：浏览器上传文件
报告错误内容：
上传文件是失败，发生以下错误

Post "http://localhost:8888/fs/63a43e1920ef271f2ed4e89a/c.txt": net/http: HTTP/1.x transport connection broken: http: ContentLength=184 with Body length 238

具体发生在向weedfs提交文件的时候。

抓包发现"github.com/imroc/req"会改变"boundary"
比如原始请求是

POST / HTTP/1.1
Host: localhost:8011
User-Agent: curl/7.77.0
Accept: */*
Content-Length: 184
Content-Type: multipart/form-data; boundary=------------------------2f17f5c764d9af7f
Connection: keep-alive

--------------------------2f17f5c764d9af7f
Content-Disposition: form-data; name="file"; filename="c.txt"
Content-Type: text/plain

=
--------------------------2f17f5c764d9af7f--

req提交的内容是

POST // HTTP/1.1
Host: localhost:8888
User-Agent: curl/7.77.0
Content-Length: 184
Accept: */*
Content-Type: multipart/form-data; boundary=c794b4a037e85e5128e14ad42dbd58bfe54f73b9eae044f020f3aca56b4e
Accept-Encoding: gzip
Connection: keep-alive

--c794b4a037e85e5128e14ad42dbd58bfe54f73b9eae044f020f3aca56b4e
Content-Disposition: form-data; name="file"; filename="c.txt"
Content-Type: application/octet-stream

=
--c794b4a037e85e5128e14ad42dbd58bfe54f73b9eae044f020f3aca56b4e

boundary由原来的"------------------------2f17f5c764d9af7"变成"c794b4a037e85e5128e14ad42dbd58bfe54f73b9eae044f020f3aca56b4e"，导致body内容长度发生改变，而头部Content-Length还保持原来的，因而不匹配，上传失败。

不确定整个0.6.0+都有问题，我有台机子是没有错误的，有待检查。