【嵌入式SD NAND】基于FATFS/Littlefs文件系統的日志框架實(shí)現
【嵌入式】基于FATFS/Littlefs文件系統的日志框架實(shí)現
1. 概述
2. 設計概要
3. 設計實(shí)現
3.1 初始化 `init`
3.2 日志寫(xiě)入 `write`
3.3 日志讀取 `read`
3.4 注銷(xiāo) `deinit`
3.5 全部代碼匯總
4. 測試
5. 總結
那么在移植好了文件系統之后,我們又應該如何應用文件系統呢?
很多人會(huì )說(shuō),這個(gè)簡(jiǎn)單,就操作文件嘛!open、read、write、close不就行了嗎!當然對于簡(jiǎn)單的使用,掌握open、read、write、close,去存儲一兩個(gè)文件或者從一兩個(gè)文件中簡(jiǎn)單的讀取下數據這確實(shí)沒(méi)有什么難度。但在實(shí)際應用中,特別是產(chǎn)品開(kāi)發(fā)過(guò)程中,往往不只是簡(jiǎn)單的操作一兩個(gè)文件就可以的,如果真是這樣,那費那么大勁移植文件系統多少有點(diǎn)浪費!
在實(shí)際項目開(kāi)發(fā)中,往往需要依托文件系統操作諸多的文件,操作諸多的數據。如通過(guò)配置文件配置機器設備信息、通過(guò)升級文件進(jìn)行產(chǎn)品升級、通過(guò)存放字庫文件實(shí)現多語(yǔ)言支持等等,這些都是比較簡(jiǎn)單的操作,讀寫(xiě)不是很頻繁,相對來(lái)說(shuō)實(shí)現比較簡(jiǎn)單,還有一類(lèi)需求讀寫(xiě)會(huì )相當頻繁,且大多數產(chǎn)品內都希望存在的,那便是日志文件,通過(guò)日志文件來(lái)記錄設備的運行數據。日志文件不同于其他功能,其往往需要具備幾個(gè)基本特性需求:
單個(gè)文件大小限制
日志總大小空間占用限制
自動(dòng)循環(huán)覆蓋
網(wǎng)上也有一些開(kāi)源的日志框架,如 Log4j,不過(guò)大都是基于 java / c ++ 實(shí)現的,雖然功能比較全面,但比較繁雜,且也難以移植應用于嵌入式開(kāi)發(fā)中。而在嵌入式開(kāi)發(fā)中,可能也受限于資源限制,并沒(méi)有發(fā)現不錯的基于文件系統的開(kāi)源日志框架(至少博主目前沒(méi)有發(fā)現,有的話(huà)歡迎大家評論區討論 ),對于如何實(shí)現一個(gè)日志框架很多人一下子可能沒(méi)有頭緒,綜上,本文將分享一個(gè)簡(jiǎn)單的基于文件系統的日志程序以供大家思考。
2. 設計概要我們需要實(shí)現的日志模塊的核心需求為:
單個(gè)文件大小限制
日志總大小空間占用限制
自動(dòng)循環(huán)覆蓋
對于一個(gè)模塊,對外僅需提供其操作的接口即可,內部的算法實(shí)現均無(wú)需對外開(kāi)放,而對于此日志模塊,對外只需提供基本的以下四個(gè)接口即可:
初始化 init
寫(xiě)日志 write
讀日志 read
注銷(xiāo) deinit
關(guān)于日志存儲的核心思想如下:
寫(xiě)數據之前先判斷當前操作的文件是否超出單個(gè)文件大小限制,如超出大小限制則進(jìn)行日志輪轉,創(chuàng )建一個(gè)新的日志文件并判斷日志文件總大小是否超出限制,如果超出則刪除最早的那一份日志文件
關(guān)于日志存儲的詳細設計如下:
日志文件格式采用:<filename>.log ,當當前文件達到單個(gè)文件大小之后,進(jìn)行文件輪轉;
假定當前限制日志每個(gè)日志文件大小為2048Byte,最多存儲10個(gè)文件;
當當前文件達到單個(gè)文件大小之后,迭代修改日志文件名:
<filename>.log -> <filename>.log.0
<filename>.log.0 -> <filename>.log.1
<filename>.log.1 -> <filename>.log.2
…
<filename>.log.8 -> <filename>.log.9
刪除 <filename>.log.9
ps:注意實(shí)際代碼操作的時(shí)候,文件修改順序是反過(guò)來(lái)的,也就是先 刪除 <filename>.log.9 再將<filename>.log.8 -> <filename>.log.9
3. 設計實(shí)現3.1 初始化 init初始化部分代碼主要功能是完成日志數據結構體的構造,并通過(guò)傳入參數 log_file_cfg_t cfg 配置日志文件的配置信息,如單個(gè)日志文件大小、日志文件名、最多存放的日志文件數等內容,日志模塊初始化部分代碼如下:
log_file_t log_storage_init(log_file_cfg_t cfg)
{
log_file_t log = NULL;
log_file_cfg_t log_cfg = NULL;
log_file_read_t log_read = NULL;
log = (log_file_t)malloc(sizeof(struct log_file_config));
if (log == NULL)
goto error;
log_cfg = (log_file_cfg_t)malloc(sizeof(struct log_file_config));
if (log_cfg == NULL) {
free(log);
log = NULL;
goto error;
}
log_read = (log_file_read_t)malloc(sizeof(struct log_file_read));
if (log_read == NULL) {
free(log);
log = NULL;
free(log_cfg);
log_cfg = NULL;
goto error;
}
memcpy(log_cfg, cfg, sizeof(struct log_file_config));
log_read->rotate_index = 0;
log_read->file_offset = 0;
log->cfg = log_cfg;
log->read = log_read;
log->user_data = NULL;
error:
return log;
}
日志寫(xiě)入部分代碼主要分為兩大部分,一部分是正常寫(xiě)入,另一部分是文件輪轉;當寫(xiě)入的文件超過(guò)單個(gè)文件大小限制時(shí),即會(huì )觸發(fā)文件輪轉操作。
在文件輪轉中,主要做的是:創(chuàng )建一個(gè)新的日志文件并判斷日志文件總大小是否超出限制,如果超出則刪除最早的那一份日志文件,具體設計細節可參考上文設計概要中的詳細設計部分。
實(shí)現代碼如下:
static int log_rotate(log_file_t log)
{
int ret = 0;
FILE *fp;
char old_filename[NAME_MAX + 10] = {0};
char new_filename[NAME_MAX + 10] = {0};
for (int i = log->cfg->rotate_num; i > 0; i --) {
memset(old_filename, 0, sizeof(old_filename));
memset(new_filename, 0, sizeof(new_filename));
snprintf(old_filename, sizeof(old_filename), i ? "%s_%d.log" : "%s.log", log->cfg->filename, i - 1);
snprintf(new_filename, sizeof(new_filename), "%s_%d.log", log->cfg->filename, i);
printf("old:%s new:%s\n", old_filename, new_filename);
if ((fp = fopen(new_filename, "r")) != NULL) {
if (fclose(fp) != 0) {
ret = -1;
goto error;
}
if (remove(new_filename) != 0) {
ret = -2;
goto error;
}
}
if ((fp = fopen(old_filename, "r")) != NULL) {
if (fclose(fp) != 0) {
ret = -1;
goto error;
}
if (rename(old_filename, new_filename) != 0) {
ret = -3;
goto error;
}
}
}
error:
return ret;
}
int log_storage_write(log_file_t log, const unsigned char *buf, unsigned int len)
{
int ret = 0;
int file_size = 0;
char full_filename[NAME_MAX + 5] = {0};
FILE *fp = NULL;
if (log == NULL || log->cfg == NULL || log->read == NULL || buf == NULL || len == 0) {
ret = -1;
goto param_error;
}
snprintf(full_filename, sizeof(full_filename), "%s.log", log->cfg->filename);
printf("fullfilename:%s\n", full_filename);
log_file_lock();
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
fseek(fp, 0L, SEEK_END);
file_size = ftell(fp);
printf("file_size:%d\n", file_size);
if ((file_size + len) > log->cfg->max_size) {
if (fclose(fp) != 0) {
ret = -3;
goto error;
}
int j = 0;
j = log_rotate(log);
printf("log rotate:%d\n", j);
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
}
if (fwrite(buf, len, 1, fp) != 1) {
fclose(fp);
ret = -4;
goto error;
}
error:
if (fp != NULL) {
if (fclose(fp) != 0) {
ret = -3;
goto error;
}
}
log_file_unlock();
param_error:
return ret;
}
此處日志讀取在本文主題中非重點(diǎn)設計內容,因此此處做簡(jiǎn)單設計,通過(guò)傳入參數判斷應該讀取哪一份文件之后進(jìn)行直接讀取。設計代碼如下:
int log_storage_read(log_file_t log, unsigned int rotate_num, unsigned char *buf, unsigned int *len)
{
int ret = 0;
int file_size = 0;
char full_filename[NAME_MAX + 5] = {0};
FILE *fp = NULL;
if (log == NULL || log->cfg == NULL || log->read == NULL || buf == NULL || len == 0) {
ret = -1;
goto param_error;
}
if (rotate_num == 0)
snprintf(full_filename, sizeof(full_filename), "%s.log", log->cfg->filename);
else
snprintf(full_filename, sizeof(full_filename), "%s.log.%d", log->cfg->filename, rotate_num);
log_file_lock();
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
/* check file length. */
fseek(fp, 0L, SEEK_END);
file_size = ftell(fp);
printf("file_size:%d\n", file_size);
if (file_size < *len)
*len = file_size;
fseek(fp, 0L, SEEK_SET);
if (fread(buf, *len, 1, fp) != 1) {
ret = -3;
fclose(fp);
goto error;
}
error:
if (fp != NULL) {
if (fclose(fp) != 0) {
ret = -4;
goto error;
}
}
log_file_unlock();
param_error:
return ret;
}
注銷(xiāo)的主要功能是將我們在 init 時(shí)創(chuàng )建的數據結構進(jìn)行回收,如果模塊內部有功能處于打開(kāi)裝填,也應關(guān)閉模塊的功能,此處我們僅需對 init 時(shí)創(chuàng )建的 log_file_t log 數據結構體進(jìn)行注銷(xiāo)、內存回收即可,具體代碼實(shí)現如下:
int log_storage_deinit(log_file_t log)
{
if (log == NULL)
return -1;
if (log->cfg != NULL)
free(log->cfg);
if (log->read != NULL)
free(log->read);
if (log->user_data != NULL)
free(log->user_data);
free(log);
return 0;
}
日志模塊內核頭文件:simple_storage.h
#ifndef __SIMPLE_STORAGE_H__
#define __SIMPLE_STORAGE_H__
#define NAME_MAX 40
struct log_file_config {
const char filename[NAME_MAX]; /* Filename of this type. */
int max_size; /* single file max size. */
int rotate_num; /* The number of files that support rotate. */
};
typedef struct log_file_config* log_file_cfg_t;
struct log_file_read {
int rotate_index; /* The rotate file index. */
int file_offset; /* The offset of the currently read file. */
};
typedef struct log_file_read* log_file_read_t;
struct log_file {
log_file_cfg_t cfg;
log_file_read_t read;
void *user_data;
};
typedef struct log_file* log_file_t;
log_file_t log_storage_init(log_file_cfg_t cfg);
int log_storage_write(log_file_t log, const unsigned char *buf, unsigned int len);
int log_storage_read(log_file_t log, unsigned int rotate_num, unsigned char *buf, unsigned int *len);
int log_storage_deinit(log_file_t log);
#endif /* __SIMPLE_STORAGE_H__ */
日志模塊內核文件: simple_storage.c
#include "simple_storage.h"
#include "simple_storage_port.h"
#include <stdio.h>
#include <string.h>
log_file_t log_storage_init(log_file_cfg_t cfg)
{
log_file_t log = NULL;
log_file_cfg_t log_cfg = NULL;
log_file_read_t log_read = NULL;
log = (log_file_t)malloc(sizeof(struct log_file_config));
if (log == NULL)
goto error;
log_cfg = (log_file_cfg_t)malloc(sizeof(struct log_file_config));
if (log_cfg == NULL) {
free(log);
log = NULL;
goto error;
}
log_read = (log_file_read_t)malloc(sizeof(struct log_file_read));
if (log_read == NULL) {
free(log);
log = NULL;
free(log_cfg);
log_cfg = NULL;
goto error;
}
memcpy(log_cfg, cfg, sizeof(struct log_file_config));
log_read->rotate_index = 0;
log_read->file_offset = 0;
log->cfg = log_cfg;
log->read = log_read;
log->user_data = NULL;
error:
return log;
}
static int log_rotate(log_file_t log)
{
int ret = 0;
FILE *fp;
char old_filename[NAME_MAX + 10] = {0};
char new_filename[NAME_MAX + 10] = {0};
for (int i = log->cfg->rotate_num; i > 0; i --) {
memset(old_filename, 0, sizeof(old_filename));
memset(new_filename, 0, sizeof(new_filename));
snprintf(old_filename, sizeof(old_filename), i ? "%s_%d.log" : "%s.log", log->cfg->filename, i - 1);
snprintf(new_filename, sizeof(new_filename), "%s_%d.log", log->cfg->filename, i);
printf("old:%s new:%s\n", old_filename, new_filename);
if ((fp = fopen(new_filename, "r")) != NULL) {
if (fclose(fp) != 0) {
ret = -1;
goto error;
}
if (remove(new_filename) != 0) {
ret = -2;
goto error;
}
}
if ((fp = fopen(old_filename, "r")) != NULL) {
if (fclose(fp) != 0) {
ret = -1;
goto error;
}
if (rename(old_filename, new_filename) != 0) {
ret = -3;
goto error;
}
}
}
error:
return ret;
}
int log_storage_write(log_file_t log, const unsigned char *buf, unsigned int len)
{
int ret = 0;
int file_size = 0;
char full_filename[NAME_MAX + 5] = {0};
FILE *fp = NULL;
if (log == NULL || log->cfg == NULL || log->read == NULL || buf == NULL || len == 0) {
ret = -1;
goto param_error;
}
snprintf(full_filename, sizeof(full_filename), "%s.log", log->cfg->filename);
printf("fullfilename:%s\n", full_filename);
log_file_lock();
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
fseek(fp, 0L, SEEK_END);
file_size = ftell(fp);
printf("file_size:%d\n", file_size);
if ((file_size + len) > log->cfg->max_size) {
if (fclose(fp) != 0) {
ret = -3;
goto error;
}
int j = 0;
j = log_rotate(log);
printf("log rotate:%d\n", j);
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
}
if (fwrite(buf, len, 1, fp) != 1) {
fclose(fp);
ret = -4;
goto error;
}
error:
if (fp != NULL) {
if (fclose(fp) != 0) {
//TODO: check the amount of disk space, delete if there is not enough space.
ret = -3;
goto error;
}
}
log_file_unlock();
param_error:
return ret;
}
int log_storage_read(log_file_t log, unsigned int rotate_num, unsigned char *buf, unsigned int *len)
{
int ret = 0;
int file_size = 0;
char full_filename[NAME_MAX + 5] = {0};
FILE *fp = NULL;
if (log == NULL || log->cfg == NULL || log->read == NULL || buf == NULL || len == 0) {
ret = -1;
goto param_error;
}
if (rotate_num == 0)
snprintf(full_filename, sizeof(full_filename), "%s.log", log->cfg->filename);
else
snprintf(full_filename, sizeof(full_filename), "%s.log.%d", log->cfg->filename, rotate_num);
log_file_lock();
fp = fopen(full_filename, "a+b");
if (fp == NULL) {
ret = -2;
goto error;
}
/* check file length. */
fseek(fp, 0L, SEEK_END);
file_size = ftell(fp);
printf("file_size:%d\n", file_size);
if (file_size < *len)
*len = file_size;
fseek(fp, 0L, SEEK_SET);
if (fread(buf, *len, 1, fp) != 1) {
ret = -3;
fclose(fp);
goto error;
}
error:
if (fp != NULL) {
if (fclose(fp) != 0) {
ret = -4;
goto error;
}
}
log_file_unlock();
param_error:
return ret;
}
int log_storage_deinit(log_file_t log)
{
if (log == NULL)
return -1;
if (log->cfg != NULL)
free(log->cfg);
if (log->read != NULL)
free(log->read);
if (log->user_data != NULL)
free(log->user_data);
free(log);
return 0;
}
在日志模塊源文件的代碼中,我們可以看到實(shí)際每次操作文件的時(shí)候,都有調用一個(gè)函數鎖操作,考慮到不同平臺的鎖操作實(shí)現不一樣,因此將此部分通過(guò)函數導出來(lái),放置在模塊的端口文件中。不同的平臺、系統根據各自的平臺和系統的情況進(jìn)行實(shí)現,如像裸機編程這類(lèi)不需要進(jìn)行鎖操作的不進(jìn)行函數實(shí)現即可。
日志模塊端口頭文件:simple_storage_port.c
#ifndef __SIMPLE_STORAGE_PORT_H__
#define __SIMPLE_STORAGE_PORT_H__
int log_file_init(void);
int log_file_lock(void);
int log_file_unlock(void);
#endif /* __SIMPLE_STORAGE_PORT_H__ */
日志模塊端口源文件:simple_storage_port.h
#include "simple_storage_port.h"
int log_file_init(void)
{
return 0;
}
int log_file_lock(void)
{
return 0;
}
int log_file_unlock(void)
{
return 0;
}
將以上代碼進(jìn)行運行測試,硬件平臺如下:
控制器: stm32f103vet6,野火指南者開(kāi)發(fā)板
存儲芯片: CS創(chuàng )世 SD nand,型號:CSNP4GCR01-AMW
文件系統: FATFS,注意此日志不受文件系統限制
操作系統: RT-Thread,此模塊與操作系統無(wú)關(guān),此處只是方便使用故自行移植了rtthread
應用層代碼如下:
int main(void)
{
/* Reset of all peripherals, Initializes the Flash interface and the Systick. */
HAL_Init();
/* USER CODE BEGIN Init */
/* USER CODE END Init */
/* Configure the system clock */
SystemClock_Config();
/* USER CODE BEGIN SysInit */
/* USER CODE END SysInit */
/* Initialize all configured peripherals */
MX_GPIO_Init();
MX_SDIO_SD_Init();
MX_USART1_UART_Init();
MX_FATFS_Init();
/* USER CODE BEGIN 2 */
struct log_file_config log_cfg = {
.filename = "test",
.max_size = 2048,
.rotate_num = 10,
};
log_file_t log = NULL;
log = log_storage_init(&log_cfg);
if (log == NULL)
return;
/* USER CODE END 2 */
/* Infinite loop */
/* USER CODE BEGIN WHILE */
unsigned char buf[2048] = {0};
int len = 0;
while (1) {
// ... 省略用戶(hù)代碼
/* 寫(xiě)入測試 */
for (int i = 0; i < 2048; i++) {
log_storage_write(log, "hello world", sizeof("hello world"));
rt_thread_mdelay(100);
}
/* 讀取測試 */
len = sizeof(buf);
memset(buf, 0, sizeof(buf));
log_storage_read(log, 1, buf, &len);
for (int i = 0; i < len; i ++)
rt_kprintf("%c", buf[i]);
rt_thread_mdelay(1000);
}
}
測試結果如下:
msg> hello worldhello world hello world hello world hello world hello world hello world hello world hello world ...省略
msh > ls
test.log 2046
test.log.0 2046
test.log.1 2046
test.log.2 2046
test.log.3 2046
test.log.4 2046
5. 總結
綜上便是基于文件系統的簡(jiǎn)易日志模塊設計的全部?jì)热萘?,雖然簡(jiǎn)陋了點(diǎn),但相信對于大部分沒(méi)有接觸過(guò)日志系統設計的人來(lái)說(shuō)提供了很好的一條設計思路。
也正因為簡(jiǎn)易,給大家對于日志系統設計的優(yōu)化留足了大量的優(yōu)化空間。比如:
文件輪轉的時(shí)候需要對每個(gè)文件的文件名進(jìn)行修改,是否可以有更好的方式不用每個(gè)文件都修改呢?
文件名的設計是不方便閱讀的,是否可以引入時(shí)間參數?
文件名設計如何引入了時(shí)間參數,當設備RTC備用電池掉電的時(shí)候又如何保證文件不會(huì )被錯誤覆蓋?
文件的讀取顯然優(yōu)化空間更大,實(shí)際上用戶(hù)不應該傳入rotate_num 參數,因為這是模塊內部的參數,用戶(hù)不可感知的
文件讀取如何做到分多次讀取一個(gè)文件的內容,且不會(huì )重復,是順序讀???
等等,以上只是我簡(jiǎn)單想到的幾點(diǎn)內容,大家不妨思考下如何實(shí)現方案更好呢?當然又還有哪些需求是需要引入的呢,也歡迎大家在評論區留言,關(guān)注我,后續抽時(shí)間再分享下改良版日志系統?。?!
*博客內容為網(wǎng)友個(gè)人發(fā)布,僅代表博主個(gè)人觀(guān)點(diǎn),如有侵權請聯(lián)系工作人員刪除。