dynamic memory management with the C language
note: cmocka has to be built separately and dynamically linked to the project
note: the CMakeLists.txt has a hardcoded library name and location assuming an Ubuntu installation
- Practice development in the C programming language.
- Learn to manage (allocate/deallocate) dynamic memory (aka memory on the heap/free store).
- Get a feel for the memory management overhead in an operating system.
- Get a basic understanding of the operation of
malloc
and the heap. - Practice working with a variety of data structures.
- Practice programming with pointers.
- Prepare for the development of the Pintos projects.
- Develop good coding style.
PA1 asks you to implement a memory pool manager which allocates dynamic memory blocks from a set pre-allocated region, mimicking the functionality of the C Standard Library function malloc()
. You are given a header file mem_pool.h
and a source file mem_pool.c
. You are not supposed to change the header file and are free to use the infrastructure in the source file to implement the user-facing functions in the header, or come up with your own.
PA1 is an assignment in the test-driven development (TDD) style. The provided main.c
driver file executes a suite of unit tests against your implementation, and your score is equal to the ratio of the number of successfully passed tests and the total number of tests. The tests use the cmocka unit test framework.
You need to submit on the course's chosen learning management system the url of your remote Github repository by the assignment deadline.
Once you fork the repository (this is your remote repository on Github, aka origin), you will clone it to your development machine (this is your local repository), and start work on it. Commit your changes to your local repository often and push them up to the remote repository occasionally. Make sure you push at least once before the due date. At the due date, your remote repository will be cloned and tested automatically by the grading script. Note: Your code should be in the master branch of your remote repository.
An autograding script will run the test suite against your files. Your grade will be based on the number of tests passed. (E.g. if your code passes 3 out of 6 test cases, your score will be 50% and the grade will be the corresponding letter grade in the course's grading scale). Note: The testing and grading will be done with fresh original copies of all the provided files. In the course of development, you can modify them, if you need to, but your changes will not be used. Only your mem_pool.c file will be used.
Your program should run on a C11 compatible compiler. Use gcc
on a Linux server for your school. The test will be run there for grading.
The assignment is due on Sun, Mar 20, at 23:59 Mountain time. The last commit to your PA1 repository before the deadline will be graded.
Free Github repositories are public so you can look at each other's code. Please, don't do that. You can discuss any programming topics and the assignments in general but sharing of solutions diminishes the individual learning experience of many people. Assignments might be randomly checked for plagiarism and a plagiarism claim may be raised against you.
Note that PA1 one is an individual assignment, not a team assignment like the upcoming Pintos assignments.
For this assignment, no external libraries should be used, except for the ANSI C Standard Library. The implementation of the data structures should be your own. We will use library implementations of data structures and programming primitives in the Pintos assignments.
Familiarize yourself with and start the following coding style guide. While you are not expected to follow every point of it, you should try to follow it enought to get a feel for what is good style and bad style. C code can quickly become unreadable and difficult to maintain.
A minimal C Reference, which should be sufficient for your needs.
The C98 Library Reference is more complete.
The C11 Standard is just provided for completeness, and you shouldn't need to read it, except peruse it out of curiosity.
Two guides for implementation of malloc()
: here and here.
- An installation as in cmocka-mem-pool.
The memory pool will work roughly like the dynamic memory management functions malloc, calloc, realloc, free
. Unlike the *alloc
functions,
- the metadata for allocated blocks will be kept in a separate dynamic memory section;
- there will be multiple independent memory pool to allocate from;
- the return value is a pointer to a
struct
which contains the memory pointer, rather than the pointer itself; - there is less hiding of (some of) the allocation metadata, to help with debugging and testing.
-
alloc_status mem_init();
This function should be called first and called only once until a corresponding
mem_free()
. It initializes the memory pool (manager) store, a data structure which stores records for separate memory pools. -
alloc_status mem_free();
This function should be called last and called only once for each corresponding
mem_init()
. It frees the pool (manager) store memory. -
pool_pt mem_pool_open(size_t size, alloc_policy policy);
This function allocates a single memory pool from which separate allocations can be performed. It takes a
size
in bytes, and an allocation policy, eitherFIRST_FIT
orBEST_FIT
. -
alloc_status mem_pool_close(pool_pt pool);
This function deallocates a single memory pool.
-
alloc_pt mem_new_alloc(pool_pt pool, size_t size);
This function performs a single allocation of
size
in bytes from the given memory pool. Allocations from different memory pools are independent. -
alloc_status mem_del_alloc(pool_pt pool, alloc_pt alloc);
This function deallocates the given allocation from the given memory pool.
-
void mem_inspect_pool(pool_pt pool, pool_segment_pt *segments, unsigned *num_segments);
This function returns a new dynamically allocated array of the pool
segments
(allocations or gaps) in the order in which they are in the pool. The number of segments is returned innum_segments
. The caller is responsible for freeing the array.Note: Fixed bug in signature:
segments
was a single pointer, and has to be double. Fixed and updated in code.
-
Memory pool (user facing)
This is the data structure a pointer to which is returned to the user by the call to
mem_pool_open
. The pointer to the allocated memory and the policy are contained in the structure, along with some allocation metadata. The user passes the pointer to the structure to the allocation/deallocation functionsmem_new_alloc
andmem_del_alloc
. The user is not responsible for deallocating the structure.Structure:
typedef struct _pool { char *mem; alloc_policy policy; size_t total_size; size_t alloc_size; unsigned num_allocs; unsigned num_gaps; } pool_t, *pool_pt;
Behavior & management:
- Passed to all functions that open, allocate on, dealocate from, and close a pool.
- The metadata contained in the structure is used by the library, so should not be overwritten by the user. It is provided for testing and debugging.
-
Allocation record (user facing)
This is the data structure a pointer to which is returned to the user for each new allocation from a given pool. Again, the pointer to the allocated memory is in this structure along with the allocated size. The user passes the pointer to the structure and the pointer to the structure of the containing memory pool to the deallocation function
mem_del_alloc
. The user is not responsible for deallocating the structure.Structure:
typedef struct _alloc { size_t size; char *mem; } alloc_t, *alloc_pt;
Behavior & management:
- Passed to the functions that allocate on and dealocate from a given pool.
- Note: A pointer to an allocation structure (aka allocation record) is the same as the pointer to the allocated memory!
-
Pool manager (library static)
This is a datastructure that the
mem_pool
library uses to store the private metadata for a single memory pool. It is hidden to the user.Structure:
typedef struct _pool_mgr { pool_t pool; node_pt node_heap; unsigned total_nodes; unsigned used_nodes; gap_pt gap_ix; unsigned gap_ix_capacity; } pool_mgr_t, *pool_mgr_pt;
Note: Notice that the user facing
pool_t
structure is at the top of the internalpool_mgr_t
structure, meaning that the two structures have the same address, and the same pointer points to both. This allows the pointer to the pool received as an argument to the allocation/deallocation functions to be cast to a pool manager pointer.Behavior & management:
- The pool manager holds pointers to all the required metadata for the memory allocations for a single pool
- The functions which make allocations in a given pool have to pass the pool as their first argument.
- The
gap_ix_capacity
is the capacity of the gap index and used to test if the index has to be expanded. If the index is expanded,gap_ix_capacity
is updated as well.
-
(Linked-list) node heap (library static)
This is packed linked list which holds nodes for all the segments (allocations or gaps) in a pool, in ascending order by memory address. That is, the first node is always going to point to the segment that starts at the beginning of the pool. This data structure is hidden from the user, except that the
num_allocs
andnum_gaps
variables in the user-facingpool_t
structure are in sync with the node heap.Structure:
typedef struct _node { alloc_t alloc_record; unsigned used; unsigned allocated; struct _node *next, *prev; // doubly-linked list for gap deletion } node_t, *node_pt;
Behavior & management:
- This is a linked list allocated as an array of
node__t
structures. If a node hasused
set to 1, it is part of the list; otherwise, it is an unused node which can be used for a new allocation. - The first node is always present and should always point to the top segment of the pool, regardless of the type of segment (allocation or gap).
- An active list node (
used == 1
) is either an allocation (allocated == 1
) or a gap (allocated == 0
). - The list is doubly-linked to simplify the deallocation of an allocated sector between two gap sectors.
- Note: Notice that the user-facing allocation record (of type
alloc_t
) is on top of the internalnode_t
, so they have the same address and a pointer to the one points to the other. Of course, the pointer has to be cast to the proper type. For example, the thealloc_pt
passed by the user as an argument to themem_new_alloc
andmem_del_alloc
has to be cast tonode_pt
before operating with the corresponding linked-list node. - The linked list is initialized with a certain capacity. If necessary, it should be resized with
realloc()
. See the correspondingstatic
function and constants in the source file.
- This is a linked list allocated as an array of
-
Gap index (library static)
This is a simple array of
gap_t
structures which holds an element for each gap that exists in a given pool and is sorted in an ascending order by size.Structure:
typedef struct _gap { size_t size; node_pt node; } gap_t, *gap_pt;
Behavior & management:
- The gap entries hold the
size
of the gaps and point to the corresponding nodes in the node heap linke list. - The array is initialized with a certain capacity. If necessary, it should be resized with
realloc()
. See the correspondingstatic
function and constants in the source file. - Use the
num_gaps
variable in the user-facingpool_t
structure as the size of the array and keep it updated. - When deleting entries from the array, pull up the entried that follow and update the size. See the corresponding
static
function. - When adding entries to the array, add at the bottom. See the corresponding
static
function. - There is a separate
static
function for sorting the array.
- The gap entries hold the
-
Pool (manager) store (library static)
This is an array of pointers to
pool_mgr_t
structures and so holds the metadata for multiple pools. See the correspondingstatic
variables and functions.Behavior & management:
- The array is initialized with a certain capacity. If necessary, it should be resized with
realloc()
. See the correspondingstatic
function and constants in the source file. - Since this array contains pointers, they can be
NULL
. The size of the array, for which astatic
variable is used, should be incremented when a new pool is opened and never decremented. The pointer to a new pool should always be added to the end of the array. When a pool is closed, the pointer should be set toNULL
.
- The array is initialized with a certain capacity. If necessary, it should be resized with
-
Pool segment (user facing)
This is a simple structure which represents a pool segment, either an allocation or a gap. Used for pool inspection by the user.
Structure:
typedef struct _pool_segment { size_t size; unsigned long allocated; } pool_segment_t, *pool_segment_pt;
Behavior & management:
- An array of such structures is returned by the function
mem_inspect_pool()
for testing, printing, and debugging. - Note: The returned array should be freed by the user.
- An array of such structures is returned by the function
The following functions are internal to the library and not exposed to the user. Their names are self-explanatory.
-
static alloc_status _mem_resize_pool_store();
If the pool store's size is within the fill factor of its capacity, expand it by the expand factor using
realloc()
. -
static alloc_status _mem_resize_node_heap(pool_mgr_pt pool_mgr);
If the node heap's size is within the fill factor of its capacity, expand it by the expand factor using
realloc()
. -
static alloc_status _mem_resize_gap_ix(pool_mgr_pt pool_mgr);
If the gap index's size is within the fill factor of its capacity, expand it by the expand factor using
realloc()
. -
static alloc_status _mem_add_to_gap_ix(pool_mgr_pt pool_mgr, size_t size, node_pt node);
Add a new entry to the gap index. The entry is gap
size
andnode
pointer to a node on the node heap of the givenpool_mgr
. -
static alloc_status _mem_remove_from_gap_ix(pool_mgr_pt pool_mgr, size_t size, node_pt node);
Remove an entry from the gap index. The entry is gap
size
andnode
pointer to a node on the node heap of the givenpool_mgr
. -
static alloc_status _mem_sort_gap_ix(pool_mgr_pt pool_mgr);
Sort the gap index in ascending order by size. Note: The index always has a length equal to the number of gaps currently in the corresponding pool.
The following variables are internal to the library and not exposed to the user. Their names are self-explanatory. They are used to hold the pool store array of pointers to pool_mgr_t
structures and are manipulated by the user-facing functions mem_init()
, mem_pool_open()
, mem_pool_close()
, and mem_free()
, and the library static function _mem_resize_pool_store()
.
static pool_mgr_pt *pool_store = NULL;
static unsigned pool_store_size = 0;
static unsigned pool_store_capacity = 0;
this section concerns future editions of the project
-
Redesign/refactor to return the memory allocation address (mem) to the user from
mem_new_alloc
instead of the allocation record address. The allocation record is embedded in the linked list node, so when the node heap is reallocated, the nodes' (and, thus, the allocation records') addresses shift. The internal infrastructure only requires an adjustment of the linked list pointers and the gap index node pointers, but the allocation record addresses the user has are invalidated. So mem should be returned and not alloc. -
Static linking of the cmocka library.