Comments (6)
@colinmarc : is this issue fixed and still kept open or its yet to be fixed ? Still getting this error though.
from hdfs.
I saw many comments saying that the file write stream is not recoverable automatically. Will the fixing be scheduled in the near future?
from hdfs.
Currently, failover during writes is sketched out, but not implemented. See this comment:
Lines 53 to 58 in cdda132
There's even a (skipped) test for it, here:
Lines 78 to 101 in cdda132
Unfortunately, the process is extremely client-heavy, so it's very complex to implement. I'll leave this issue open to track it.
To answer your question, you should indeed close the stream and start a new write.
from hdfs.
I think this package is the 'native' hdfs client package for golang and it will be very nice if you can add the failover recovery part.
For the retrying, do you mean to close the same file and use Append() to get a new stream or create another file?
By the way, if the write fails, how can I ensure that the Close() will succeed? If I fails to Close() I'll lost the data not written to hdfs
from hdfs.
Hi, @colinmarc . I took your advice and use reopening to retry, but got the following error constantly:
2017/08/25 03:26:00.320699 file_writer.go:108: [INFO] Close error:updateBlockForPipeline call failed with ERROR_APPLICATION (java.io.IOException) file:hdfs://rndhadoop001.rnd.fwmrm.net:8020/user/am/scan_task/2017-08-25/10.2.3.85_advanced_m/user-bak002-20170825032137.log
2017/08/25 03:26:00.321857 file_writer.go:114: [WARNING] Reopen error:append /user/am/scan_task/2017-08-25/10.2.3.85_advanced_m/user-bak002-20170825032137.log: append call failed with ERROR_APPLICATION (org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException) file:hdfs://rndhadoop001.rnd.fwmrm.net:8020/user/am/scan_task/2017-08-25/10.2.3.85_advanced_m/user-bak002-20170825032137.log
the retry logic:
for {
_, err = w.hdfsWriter.Write(b)
if nil == err {
break
}
retry := 0
lastSleepTime := 0
closeOpenRetry := 0
for nil != err && retry < MAX_WRITE_RETRY {
LogWarning(WARN, INTERNAL_LOGIC, "Error:%s retry:%s", err.Error(), w.name)
lastSleepTime = GetRetrySleepTime(BUSY_RETRY, lastSleepTime)
time.Sleep(time.Duration(lastSleepTime) * time.Second)
_, err = w.hdfsWriter.Write(b)
retry++
}
for nil != err && closeOpenRetry < MAX_CLOSE_OPEN_RETRY {
LogWarning(WARN, INTERNAL_LOGIC, "Error:%s closeopenretry:%s", err.Error(), w.name)
lastSleepTime = GetRetrySleepTime(BUSY_RETRY, lastSleepTime)
time.Sleep(time.Duration(lastSleepTime) * time.Second)
err = w.hdfsWriter.Close()
if nil != err {
LogInfo("Close error:%s file:%s", err.Error(), w.name)
}
tFn := strings.TrimPrefix(w.name, HDFS_FILE_PREFIX)
protocol := tFn[:strings.Index(tFn, "/")]
tmpWriter, err := GetHdfsClient(protocol).Append(w.name[len(HDFS_FILE_PREFIX)+len(protocol):])
if nil != err {
LogWarning(WARN, INTERNAL_LOGIC, "Reopen error:%s file:%s", err.Error(), w.name)
} else {
w.hdfsWriter = tmpWriter
}
}
}
the GetHdfsClient() will get hdfs.Client and w.hdfsWriter is hdfs.FileWriter
Can you help me on this? Thans a lot
from hdfs.
@colinmarc since this feature is not currently implemented, and seems relatively easy to implement the basic logic in my own application, I was hoping you could expand a little more on @kof02guy's question regarding closing the open file to retry.
To answer your question, you should indeed close the stream and start a new write.
Based on the BlockWriter comment, how would one "close the stream" in the most correct way? I would imagine calling Close on that current FileWriter would fail due to the same error that caused Write to fail.
from hdfs.
Related Issues (20)
- Is it thread-safe that NamenodeConnection?
- could not find "github.com/colinmarc/hdfs/v2/protocol/hadoop_common" "github.com/colinmarc/hdfs/v2/security"
- support UserGroupInformation
- EC not working properly HOT 1
- Allow setting CreateFlag when creating files
- Namenodes not been filtered by fs.defaultFS
- the dns_canonicalize_hostname value not being respected
- panic: mkdir /demo: read tcp 192.168.10.1:57318->192.168.10.102:9870: wsarecv: An established connection was aborted by the software in your host machine.
- how to do setQuota
- Empty file after CopyToRemote HOT 1
- Readme suggestion: also show memory usage advantage HOT 2
- Why the return of func Read() with a FileReader is not consistent?
- Downloading the encrypted file HOT 2
- Libhdfs.so bindings for hdfs-go? HOT 1
- hadoop3 - write: broken pipe
- An error is reported when reading a file in the HAR package
- Support of hadoop delegation tokens with HADOOP_TOKEN_FILE_LOCATION env
- no available namenodes: SASL handshake: wrong Token ID. Expected 0504, was 607d HOT 1
- How can I Remove file into Trash instead of removing files permanently? HOT 1
- Walk on empty directory will return io.EOF error HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from hdfs.