hashdatainc / gopherwood Goto Github PK

1. when evict block to oss, should check whether the evicted block is the last bucket of the local space, if it is the last bucket of the local space, it size may not equal BUCKET_SIZE;

2. when open->write->close the file many times, the gopherwood may throw the error that "too many files open", maybe some file descriptor is not closed;

Currently, with the simple implementation. The read latency would be high because there might be many blocks need to be loaded.
A common way to improve the read performance is to add multi-thread preload mechanism.

the caller seems can not get file stat for example file size and attrs etc. directly , please provide it

workfile - logtape may seek exceed file size, so need enhance seek

When a process communicated with shared memory for an file operation. It should remember the acquired active blocks in it-selves memory to eliminate shared memory contention.

1. test code

#include
#include
#include <fcntl.h>
#include
#include
#include "gopherwood/gopherwood.h"

int main(int agrInt, char **agrStr)
{
for (int i = 0; i < 10; i++) {
testGWWrite("TestReadWriteSeek-ReadEvictBlock");
}

return 0

}

using namespace std;

void testGWWrite(std::string fileName) {
AccessFileType type = randomType;

gopherwoodFS gwFS = gwCreateContext((char *) fileName.c_str());
gwFile file = gwOpenFile(gwFS, (char *) fileName.c_str(), O_CREAT);

int SIZE = 128;
//3. construct the file name
std::stringstream ss;
ss << "/ssdfile/ssdkv/" << fileName;
std::string filePath = ss.str();

//4. read data from file
std::ifstream infile;
infile.open(filePath.c_str());


int totalWriteLength = 0;
char *buf = new char[SIZE];
infile.read(buf, SIZE);
int readLengthIn = infile.gcount();
while (readLengthIn > 0) {
    totalWriteLength += readLengthIn;
    std::cout << "totalWriteLength=" << totalWriteLength << ",readLength="
              << readLengthIn << std::endl;
    std::cout << "buf=" << buf << std::endl;
    //5. write data to the gopherwood
    gwWrite(gwFS, file, buf, readLengthIn);

    buf = new char[SIZE];
    infile.read(buf, SIZE);
    readLengthIn = infile.gcount();
}

gwCloseFile(gwFS, file);

std::cout << "*******END OF WRITE*****, totalWriteLength=" << totalWriteLength << std::endl;

}
2. pre-condition
code version:
branch: master
commit e385461
Author: houliang [email protected]
Date: Wed Feb 7 16:58:10 2018 +0800

running env:
/ssdfile/ssdkv/TestReadWriteSeek-ReadEvictBlock 4096 bytes
/ssdfile/ssdkv/sharedMemory/smFile 40960 bytes and initalize to total zero

3. assert exception
   call stack:
   #0 0x00007ffff6c1a1f7 in raise () from /lib64/libc.so.6
   #1 0x00007ffff6c1b8e8 in abort () from /lib64/libc.so.6
   #2 0x00007ffff6c13266 in __assert_fail_base () from /lib64/libc.so.6
   #3 0x00007ffff6c13312 in __assert_fail () from /lib64/libc.so.6
   #4 0x00007ffff7b6f389 in Gopherwood::Internal::FileSystemImpl::acquireNewBlock (this=0x605a70, fileName=0x607368 "TestReadWriteSeek-ReadEvictBlock") at /opt/Gopherwood/src/core/FileSystemImpl.cpp:533
   #5 0x00007ffff7b84692 in Gopherwood::Internal::OutputStreamImpl::writeInternal (this=0x608b10, buf=0x60b900 "", size=128) at /opt/Gopherwood/src/core/OutputStreamImpl.cpp:101
   #6 0x00007ffff7b84413 in Gopherwood::Internal::OutputStreamImpl::write (this=0x608b10, buf=0x60b900 "", size=128) at /opt/Gopherwood/src/core/OutputStreamImpl.cpp:72
   #7 0x00007ffff7b83ae5 in Gopherwood::OutputStream::write (this=0x607330, buf=0x60b900 "", size=128) at /opt/Gopherwood/src/core/OutputStream.cpp:32
   #8 0x00007ffff7b8acc7 in gwWrite (fs=0x6072f0, file=0x607310, buffer=0x60b900, length=128) at /opt/Gopherwood/src/core/gopherwood.cpp:313
   #9 0x0000000000401633 in testGWWrite (fileName="TestReadWriteSeek-ReadEvictBlock") at TestGWAPI.cpp:48
   #10 0x0000000000401c2a in main (agrInt=2, agrStr=0x7fffffffe258) at TestGWAPI.cpp:128

Currently if gopherwood is formatted, the previously opened gwContext might got error because it was not informed of the format operation.

A simple proposal could be archive the SharedMemory information and the corresponding identifier to SMLock file. Then a checking mechanism could be built based on this information.

export interface should confirm C style and use extern 'c' to brace, if not, c style caller complie error

Currently, the OSS file format is something like filepath_blocknum. This format will got conflict problem when different hosts evicting same block of filepath.

To solve this problem, we need to add unique OSS file prefix for different hosts.

Currently, we support multi-write, but there is still one problem.

Suppose two writer ActiveStatus opened, they don't know each other's Eof if no gwFlush was issued. To overcame this, we need to update block Eof once we finished a gwWrite operation or switched to next block. By doing this, any opening thread will know the updated Eof by checking the SharedMemBucket Eof field.

gwSeek definition is:

int gwSeek(gopherwoodFS fs, gwFile file, tOffset desiredPos);

(1) seems can not apply to SEEK_END and SEEK_CUR
(2) return value is not the current offset from file begin, maybe caller would rewrite calling code

1. healthy check
2. ActiveStatus alive check
3. Backend block eviction if exceed the threshold.

if gopherwood use oss, for performance, please provide 'cancelfile', the caller not need to call
close->delete flow and just call cancelfile is ok while transaction is break

imply: the cancelfile would call oss cancelfile , please refer to oss

current file delete interface is
int gwDeleteFile(gopherwoodFS fs, gwFile file);
it is not convenient for using, please design it like following:
int gwDeleteFile(char* filePath);

Currently we use a fixed formula to calculate quota size when an ActiveStatus acquire new blocks.
As we planned, the enhancement to dynamically adjust quota size should be implemented after function test was 100% written.

`gopherwoodFS gwCreateContext(char *workDir, GWContextConfig *config) {
LOG(Gopherwood::Internal::INFO, "------------------gwCreateContext start------------------");
gopherwoodFS retVal = NULL;

if (config != NULL) {
    Configuration::NUMBER_OF_BLOCKS = config->numBlocks;
    Configuration::LOCAL_BUCKET_SIZE = config->blockSize;
    Configuration::CUR_CONNECTION = config->numPreDefinedConcurrency;
}`

when init the context, should check the config value is NULL or not, if the config.* is null, then the context will be in a wrong status.

if user only get the lib and not initialize anything, it would report errors, for example not build
and initialize smFile etc, please find a way to do it automatically, not need user to do it repeatly