Comments (7)
I also have the same problem, already tried all python libraries available.
Did anyone solve this with a magical workaround?
from hadoop-docker.
Not sure how this would translate if using docker-compose but can get this to work using:
docker run -h localhost -p 50070:50070 -p 50075:50075 <<Container_Name>>
from hadoop-docker.
I was trying to use webhdfs too and found a problem. In my case the problem is that webhdfs redirect to a data node every time i try to write to a file. And it seems that the redirection URL use the internal machine name of the docker image (something like a65ec753065c). Any idea about this?
The following is an example request:
curl -i -X PUT -T ~/Downloads/JEA_BLOWER_DEFINITION.csv "http://localhost:50070/webhdfs/v1/user/root/f.txt?op=CREATE&user.name=root&overwrite=true"
HTTP/1.1 100 Continue
HTTP/1.1 307 TEMPORARY_REDIRECT
Cache-Control: no-cache
Expires: Wed, 19 Oct 2016 03:45:02 GMT
Date: Wed, 19 Oct 2016 03:45:02 GMT
Pragma: no-cache
Expires: Wed, 19 Oct 2016 03:45:02 GMT
Date: Wed, 19 Oct 2016 03:45:02 GMT
Pragma: no-cache
Set-Cookie: hadoop.auth="u=root&p=root&t=simple&e=1476884702571&s=n+WgHqacT3Q5OthGXHXPBtD2YlQ="; Path=/; Expires=Wed, 19-Oct-2016 13:45:02 GMT; HttpOnly
Location: http://a65ec753065c:50075/webhdfs/v1/user/root/f.txt?op=CREATE&user.name=root&namenoderpcaddress=a65ec753065c:9000&overwrite=true
Content-Type: application/octet-stream
Content-Length: 0
Server: Jetty(6.1.26)
from hadoop-docker.
I am having the same issue as above, will it be addressed soon?
from hadoop-docker.
@PhilipMourdjis if you're using docker-compose you can put hostname localhost like this:
hadoop:
image: <image_name>
hostname: localhost
ports:
- 50070:50070
- 50075:50075
from hadoop-docker.
Just follow the redirect message Location
from hadoop-docker.
Notice:Step 2: Submit another HTTP PUT request using the URL in the Location header (or the returned response in case you specified noredirect) with the file data to be written.
FYI Link
from hadoop-docker.
Related Issues (20)
- Can't write to hdfs HOT 2
- Yum Checksum Issue When Updating `libselinux` HOT 6
- call from sqoop to haoop : Connection refused
- Error starting MRAppMaster java.lang.UnsatisfiedLinkError:
- container is not running hadoop with localhost 8020 as default namenode url HOT 1
- Data volumes for persistence and connect to Hive HOT 2
- Image published to the public Docker registry is outdated
- Port Forwarding Hadoop Job Monitor URL HOT 1
- Docker pull leads to not authorized error
- Seems that the JDK link is unavailable HOT 2
- WebHdfsFileSystem source version 2.7.1 mismtach ??
- Error writing file HOT 2
- Hadoop 2.7.1 Link is Unavailable
- Get error when run bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar grep input output 'dfs[a-z.]+' HOT 2
- Unable to install wget: Rpmdb checksum is invalid: dCDPT(pkg checksums): wget... HOT 4
- Impossible to run haddop HOT 1
- install hadoop docker on multi hosts
- Site unreachable with Hadoop on Docker HOT 3
- Change default value of dfs.encrypt.data.transfer.cipher.suites to AES?
- Nothing done when run bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.1.jar grep input output 'dfs[a-z.]+'
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hadoop-docker.