moya-lang / allocator Goto Github PK

In the documentation, under "Design decisions", you state that the standard allocator is used. I've been testing with map, and it appears this is the case. So when I run the benchmark, the performance of the default allocator and this one is the same.

So I am a bit unclear on why you chose to do this? It seems you are not even using the allocator you wrote and are instead just defaulting to the std one?

Does the decision have anything to do with this?

#2

Also, what is the reasoning behind this:

template <class U>
Allocator(const Allocator<U, growSize>& other)
{
	if (!std::is_same<T, U>::value)
		defaultAllocator = new std::allocator<T>();
}

Why do you use the standard allocator if U and T are different? This appears to be called from rebind. Perhaps I'm a bit clueless on this since I'm new to allocators.

I am building on MSVC 2017 as well.

I would really like to get this working as it seems like a very elegant and simple way to boost map performance.

Thanks.

I really miss move constructor and move assignment operator (they've been deleted). My code has something like this:

class my_class {
Moya::MemoryPool poo;
};

my_class create() {
my_class m;
...
return m;
}

my_class x = create(); // This doesn't compile because move constructor is deleted.

Can you fix it?

Modified Measure.cpp to include a pushback test for std::vector

#include

PushBackTest<std::vector<DataType>> pushBackVectorTestStl;
PushBackTest<std::vector<DataType, MemoryPoolAllocator>> pushBackVectorTestFast;
printTestStatus("Vector PushBack", pushBackVectorTestStl, pushBackVectorTestFast);

At run time this fails with

terminate called after throwing an instance of 'std::bad_alloc'
what(): std::bad_alloc
Aborted (core dumped)

This code fail when compile with g++, but it works well with clang++.

string hello = "hello";
list<char, Moya::Allocator<char>> s;
s.insert(s.end(), hello.begin(), hello.end());
// but it works if you do s.insert(s.end(), 'a')

Here is the error message,

In file included from prog_joined.cpp:1:
In file included from ./precompiled/headers.h:48:
In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/list:63:
/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/stl_list.h:933:16: error: no matching conversion for functional-style cast from 'const std::__cxx11::_List_base<char, Moya::Allocator<char, 1024>>::_Node_alloc_type' (aka 'const Allocator<std::_List_node<char>, 1024UL>') to 'std::__cxx11::list<char, Moya::Allocator<char, 1024>>::allocator_type' (aka 'Moya::Allocator<char, 1024>')
      { return allocator_type(_Base::_M_get_Node_allocator()); }
               ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/list.tcc:137:30: note: in instantiation of member function 'std::__cxx11::list<char, Moya::Allocator<char, 1024>>::get_allocator' requested here
        list __tmp(__first, __last, get_allocator());
                                    ^
Line 185: Char 11: note: in instantiation of function template specialization 'std::__cxx11::list<char, Moya::Allocator<char, 1024>>::insert<__gnu_cxx::__normal_iterator<char *, std::__cxx11::basic_string<char>>, void>' requested here
        s.insert(iter, text.begin(), text.end());
          ^
Line 82: Char 7: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'const Allocator<std::_List_node<char>, [...]>' to 'const Allocator<char, [...]>' for 1st argument
class Allocator : private MemoryPool<T, growSize>
      ^
Line 82: Char 7: note: candidate constructor (the implicit default constructor) not viable: requires 0 arguments, but 1 was provided

I try to fix it by myself but fail. could you help to have a look?

It always gives a compile error

example:
https://godbolt.org/z/b7e1WaWdK

Hi,
I got below error when compile with make command, would you please help check it?
g++ --std=c++17 -Wall Measure.cpp -O3 -o Measure
In file included from /usr/include/c++/9/map:61,
from Measure.cpp:6:
/usr/include/c++/9/bits/stl_map.h: In instantiation of ‘class std::map<int, int, std::less, Moya::Allocator<int, 1024> >’:
Measure.cpp:24:19: required from ‘class PerformanceTest<std::map<int, int, std::less, Moya::Allocator<int, 1024> > >’
Measure.cpp:81:7: required from ‘class MapTest<std::map<int, int, std::less, Moya::Allocator<int, 1024> > >’
Measure.cpp:140:85: required from here
/usr/include/c++/9/bits/stl_map.h:122:71: error: static assertion failed: std::map must have the same value_type as its allocator
122 | static_assert(is_same<typename _Alloc::value_type, value_type>::value,
| ^~~~~
make: *** [Makefile:2: default,] Error 1

It is very slow compared to std::allocator. In any case, the rules on how to create allocators are now as the standard requires. This means to compile in DEBUG mode on MSVC one needs to make sure MSVC std lib iterators are not built in a debug mode.

#ifdef _ITERATOR_DEBUG_LEVEL
#if _ITERATOR_DEBUG_LEVEL == 2
#error _ITERATOR_DEBUG_LEVEL must be set to 0
#endif
#endif

#include <memory>

namespace Moya {

    namespace detail {
        template <class T, std::size_t growSize = 1024>
        class MemoryPool
        {
            struct Block
            {
                Block* next{};
            };

            class Buffer
            {
                static const std::size_t blockSize = sizeof(T) > sizeof(Block) ? sizeof(T) : sizeof(Block);
                uint8_t data[blockSize * growSize]{};

            public:

                Buffer* const next{};

                Buffer(Buffer* next) :
                    next(next)
                {
                }

                T* getBlock(std::size_t index)
                {
                    return reinterpret_cast<T*>(&data[blockSize * index]);
                }
            };

            Block* firstFreeBlock = nullptr;
            Buffer* firstBuffer = nullptr;
            std::size_t bufferedBlocks = growSize;


        public:

            MemoryPool() = default;
            MemoryPool(MemoryPool&& memoryPool) = delete;
            MemoryPool(const MemoryPool& memoryPool) = delete;
            MemoryPool operator =(MemoryPool&& memoryPool) = delete;
            MemoryPool operator =(const MemoryPool& memoryPool) = delete;

            ~MemoryPool()
            {
                while (firstBuffer) {
                    Buffer* buffer = firstBuffer;
                    firstBuffer = buffer->next;
                    delete buffer;
                }
            }

            T* allocate()
            {
                if (firstFreeBlock) {
                    Block* block = firstFreeBlock;
                    firstFreeBlock = block->next;
                    return reinterpret_cast<T*>(block);
                }

                if (bufferedBlocks >= growSize) {
                    firstBuffer = new Buffer(firstBuffer);
                    bufferedBlocks = 0;
                }

                return firstBuffer->getBlock(bufferedBlocks++);
            }

            void deallocate(T* pointer)
            {
                Block* block = reinterpret_cast<Block*>(pointer);
                block->next = firstFreeBlock;
                firstFreeBlock = block;
            }
        };
    } // detail

    ///--------------------------------------------------------------------------------------------------------------
    template <class T, std::size_t growSize = 1024>
    class Allocator final :
        private detail::MemoryPool<T, growSize>,
        public std::allocator<T>
    {
        using memory_pool_type = detail::MemoryPool<T, growSize>;

    public:

        using parent = std::allocator<T>;
        using parent::parent;

        template <class U>
        struct rebind
        {
            typedef Allocator<U, growSize> other;
        };


        Allocator() {}

        Allocator(Allocator& allocator) :
            copyAllocator(&allocator)
        {
        }

        template <class U>
        Allocator(const Allocator<U, growSize>& other)
        {
        }

        ~Allocator()
        {
        }

        using pointer = typename parent::pointer;
        using size_type = typename parent::size_type;

        pointer allocate(size_type n)
        {
            if (n < 1)
                return nullptr;

            return memory_pool_type::allocate();
        }

        void deallocate(pointer p, size_type n)
        {
            memory_pool_type::deallocate(p);
        }
    };
} // Moya

Why the vector is not on this list???

The Allocator requires to have one instance per each used container. The new MSVC STL implementation uses one allocator for all instances of a container type. As this is not allowed, therefore the copy construction for the Allocator has been deleted. The library keeps working on other compilers. I am very open to hear any ideas how to solve the issue. Thanks!

make
g++ --std=c++17 -Wall Measure.cpp -O3 -o Measure
Measure.cpp: In instantiation of ‘void MapTest::testIteration(size_t) [with Container = std::map<long unsigned int, long unsigned int, std::less, Moya::Allocator<std::pair<const long unsigned int, long unsigned int>, 1024> >; size_t = long unsigned int]’:
Measure.cpp:87:18: required from here
Measure.cpp:92:57: error: dependent-name ‘Container::value_type’ is parsed as a non-type, but instantiation yields a type
92 | this->container.insert(Container::value_type(size, size));
| ~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~
Measure.cpp:92:57: note: say ‘typename Container::value_type’ if a type is meant
Measure.cpp: In instantiation of ‘void MapTest::testIteration(size_t) [with Container = std::map<long unsigned int, long unsigned int>; size_t = long unsigned int]’:
Measure.cpp:87:18: required from here
Measure.cpp:92:57: error: dependent-name ‘Container::value_type’ is parsed as a non-type, but instantiation yields a type
Measure.cpp:92:57: note: say ‘typename Container::value_type’ if a type is meant
Makefile:2: recipe for target 'default,' failed
make: *** [default,] Error 1

When I test it for std::vector, it will fail in construct()

It says in the README that the allocator is threadsafe but I cannot see how this could be the case.

I am looking for something I can use with MS concurrent_unordered_map.

Example:

struct M
{
...
};
typedef Moya::Allocator<std::pair<const int,M>, 1024*1024> MemoryPoolAllocator;
std::unordered_multimap<int,M,std::hash<int>,std::equal_to<int>, MemoryPoolAllocator> evs;

Still uses new/delete. It fails there:

            if (!std::is_same<T, U>::value)
                rebindAllocator = new std::allocator<T>();

Is there a reason you use std::allocator<void>::const_pointer instead of just const void*? In C++17, std::allocator<void> is deprecated and it will be completely gone in C++20.