hzhang/HangmanLab.Backend
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

add: backup version converter/ backup logic 1.0

Browse Source
This commit is contained in:
h z
2025-04-25 13:00:41 +01:00
parent 35c8934963
commit 84494827ad
15 changed files with 2107 additions and 76 deletions
Download Patch File Download Diff File Expand all files Collapse all files

818
api/backup.py
View File

@@ -1,28 +1,363 @@
"""
Backup and Restore System
=========================
This module provides functionality for creating backups of the application's data and restoring from those backups.
Backup Structure
---------------
The backup is a zip file containing the following structure:
- Root/
- tree/ # Contains the entire path hierarchy and markdown files
- .json.meta # Metadata for the root path
- [markdown].json # Markdown files directly under root
- [folder]/ # Subfolders representing paths
- .json.meta # Metadata for this path
- [markdown].json # Markdown files in this path
- template/ # Contains markdown templates
- [template].json # One JSON file per template, using title as filename
- webhook.json # Contains all webhooks and their settings
Data Format
-----------
1. Markdown files (.json):
{
"title": "...", # Title of the markdown
"content": "...", # Content of the markdown
"created_at": "...", # Creation timestamp
"order": "...", # Order value for sorting
"shortcut": "...", # Shortcut value if any
"backup_id": ..., # Reference ID for the backup
"setting": { # Optional settings object
"permission_setting": "...", # Permission value (private, protected, etc.)
"template_setting": { # Template settings if any
"template_id": ..., # Template ID reference
"template_ref": { # Reference to the template
"title": "..." # Title of the template for lookup
}
}
}
}
2. Path metadata (.json.meta):
{
"name": "...", # Name of the path
"order": "...", # Order value for sorting
"backup_id": ..., # Reference ID for the backup
"webhook_setting": { # Optional webhook settings
"recursive": true/false, # Whether webhook applies to subpaths
"additional_header": "...", # Additional headers for webhook
"enabled": true/false, # Whether webhook is enabled
"on_events": ... # Event types that trigger the webhook
},
"webhook_ref": { # Reference to the webhook
"hook_url": "..." # URL of the webhook for lookup
},
"template_setting": { # Template settings if any
"template_id": ... # Template ID reference
},
"template_ref": { # Reference to the template
"title": "..." # Title of the template for lookup
}
}
3. Template files (template/[name].json):
{
"title": "...", # Title of the template
"parameters": {...}, # Parameters for the template
"layout": "..." # Layout content of the template
}
4. Webhook file (webhook.json):
[
{
"backup_id": ..., # Reference ID for the backup
"hook_url": "...", # URL of the webhook
"settings": [ # Array of settings for this webhook
{
"recursive": true/false, # Whether webhook applies to subpaths
"additional_header": "...", # Additional headers for webhook
"enabled": true/false, # Whether webhook is enabled
"on_events": ... # Event types that trigger the webhook
}
]
}
]
How to Add New Information to Backup
-----------------------------------
To add new information to the backup system, follow these steps:
1. Adding a new field to an existing entity:
- For Path entities: Add the field to the path metadata in the traverse() function
- For Markdown entities: Add the field to the md_data dictionary in the traverse() function
- For Templates: Add the field to the template_dict in export_templates()
- For Webhooks: Add the field to the webhook_entry in export_webhooks()
2. Adding a new entity type:
- Create a new export_[entity]() function similar to export_webhooks() or export_templates()
- Call this function from get_backup()
- Create a corresponding import_[entity]() function for restoration
- Call this function from load_backup()
3. Example: Adding a new "tags" field to markdown:
In the traverse() function, modify the md_data creation:
md_data = {
"title": md.title,
"content": md.content,
"created_at": md.created_at,
"order": md.order,
"shortcut": md.shortcut,
"backup_id": md.id,
"tags": md.tags # New field
}
Then in process_markdown_file(), handle the new field:
tags = md_data.get("tags", []) # Get tags with default empty list
# Later when creating/updating the markdown:
if existing_md:
existing_md.tags = tags
else:
new_md = Markdown(
# other fields...
tags=tags
)
4. Example: Adding a new entity type "Comments":
Create export function:
def export_comments():
# Export all comments to comments.json file in the root directory
with get_db() as session:
comments = session.query(Comment).all()
comment_data = []
for comment in comments:
comment_dict = comment.to_dict()
# Process and add to comment_data
with open('comments.json', 'w') as f:
json.dump(comment_data, f, default=str, indent=2)
Call it from get_backup():
# After other exports
export_comments()
Create import function:
def import_comments(comments_file, session):
# Logic to import comments
Call it from load_backup():
# After other imports
import_comments(os.path.join(root_dir, "comments.json"), session)
Maintaining Backward Compatibility
---------------------------------
When adding new fields or entities:
1. Always use .get() with default values when reading JSON data
2. Check if fields exist before accessing them
3. Handle both old and new formats in import functions
4. Use conditional logic to process data based on available fields
5. Keep the basic structure of the backup intact
For example, in process_markdown_file():
# Handle both old and new formats
if "setting" in md_data:
# Process new format
else:
# Process old format for backward compatibility
ID Handling
----------
The backup system maintains its own ID references:
1. Database IDs are not directly used in the backup
2. Each entity gets a backup_id for reference within the backup
3. When restoring, new database IDs are generated
4. References between entities use lookup by natural keys (e.g., title, URL)
"""
import shutil
from datetime import datetime
import tempfile
import zipfile
from flask import Blueprint, send_file, jsonify, request
import os
import json
from api import require_auth
from db import get_db
from db.models.Markdown import Markdown
from db.models.Path import Path
from db.models.MarkdownSetting import MarkdownSetting
from db.models.MarkdownTemplateSetting import MarkdownTemplateSetting
from db.models.MarkdownPermissionSetting import MarkdownPermissionSetting
from db.models.MarkdownTemplate import MarkdownTemplate
from db.models.PathSetting import PathSetting
from db.models.WebhookSetting import WebhookSetting
from db.models.PathTemplate import PathTemplate
from db.models.Webhook import Webhook
import threading
import logging
logger = logging.getLogger(__name__)
backup_bp = Blueprint('backup', __name__, url_prefix='/api/backup')
def check_and_convert_backup_version(backup_dir):
"""
Check the backup version and convert it if necessary.
Args:
backup_dir (str): Path to the backup directory
Returns:
tuple: (success, error_response)
- success (bool): True if the check and conversion was successful, False otherwise
- error_response: None if successful, otherwise a Flask response object with an error message
"""
from misc.backup_converters import get_backup_version, CURRENT_VERSION, convert_backup
backup_version = get_backup_version(backup_dir)
if backup_version != CURRENT_VERSION:
logger.info(f"Converting backup from version {backup_version} to {CURRENT_VERSION}")
try:
convert_backup(backup_dir, CURRENT_VERSION)
return True, None
except ValueError as e:
logger.error(f"Failed to convert backup: {e}")
return False, jsonify({"error": f"Failed to convert backup: {e}"}), 400
return True, None
@backup_bp.route('/convert', methods=['POST'])
@require_auth(roles=['admin'])
def convert_backup_endpoint():
"""
Convert an old version backup to the current version format.
This endpoint accepts an uploaded backup file, converts it to the current version format,
and returns the converted backup file as an attachment. The conversion process handles the differences
between different backup formats, including the directory structure, file formats, and metadata.
Request:
- file: The backup file to convert (multipart/form-data)
Returns:
The converted backup file as an attachment.
Response Codes:
- 200: Conversion successful
- 400: No file provided or invalid file
- 429: Another backup operation is in progress
- 500: Conversion failed
"""
if not backup_lock.acquire(blocking=False):
return jsonify({"error": "Another backup operation is in progress. Please try again later."}), 429
try:
if 'file' not in request.files:
return jsonify({"error": "No file provided"}), 400
uploaded_file = request.files['file']
temp_dir = tempfile.mkdtemp()
backup_dir = os.path.join(temp_dir, "backup")
os.makedirs(backup_dir)
zip_path = os.path.join(temp_dir, "backup.zip")
uploaded_file.save(zip_path)
with zipfile.ZipFile(zip_path, 'r') as zip_ref:
zip_ref.extractall(backup_dir)
success, error_response = check_and_convert_backup_version(backup_dir)
if not success:
return error_response
timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
archive_name = f"converted_backup_{timestamp}"
archive_path = shutil.make_archive(
base_name=os.path.join(temp_dir, archive_name),
format='zip',
root_dir=backup_dir
)
shutil.rmtree(backup_dir)
return send_file(
archive_path,
as_attachment=True,
download_name=f"{archive_name}.zip",
)
except Exception as e:
logger.error(f"Failed to convert backup: {e}")
return jsonify({"error": f"Failed to convert backup: {e}"}), 500
finally:
backup_lock.release()
backup_lock = threading.Lock()
@backup_bp.route('/', methods=['GET'])
@require_auth(roles=['admin'])
def get_backup():
"""
Create a backup of the application's data.
This function creates a backup of the application's data, including:
- The tree structure (paths and markdowns)
- Templates
- Webhooks
- Version information
The backup is returned as a zip file attachment.
Returns:
A zip file containing the backup data.
Response Codes:
- 200: Backup created successfully
- 500: Failed to create backup
"""
try:
if os.path.exists('Root'):
shutil.rmtree('Root')
os.makedirs('Root')
os.chdir('Root')
os.makedirs('tree')
os.makedirs('template')
from misc.backup_converters import CURRENT_VERSION
with open('version.json', 'w') as f:
json.dump({"version": CURRENT_VERSION}, f, indent=2)
export_webhooks()
export_templates()
paths = {}
with get_db() as session:
pths = session.query(Path).all()
paths = {p.id : p for p in pths}
os.chdir('tree')
traverse(1, paths)
os.chdir('..')
os.chdir('..')
timestamp = datetime.now().strftime('%Y-%m-%d_%H-%M-%S')
archive = shutil.make_archive(base_name=timestamp, format='zip', root_dir='Root')
@@ -45,32 +380,165 @@ def create_and_cd(path_name):
def cd_back():
os.chdir('..')
def export_webhooks():
with get_db() as session:
webhooks = session.query(Webhook).all()
webhook_data = []
for webhook in webhooks:
webhook_dict = webhook.to_dict()
backup_id = len(webhook_data) + 1
webhook_settings = session.query(WebhookSetting).filter_by(webhook_id=webhook.id).all()
settings_list = []
for setting in webhook_settings:
setting_dict = setting.to_dict()
setting_dict.pop('id', None)
setting_dict.pop('webhook_id', None)
settings_list.append(setting_dict)
webhook_entry = {
'backup_id': backup_id,
'hook_url': webhook_dict['hook_url'],
'settings': settings_list
}
webhook_data.append(webhook_entry)
with open('webhook.json', 'w') as f:
json.dump(webhook_data, f, default=str, indent=2)
def export_templates():
with get_db() as session:
templates = session.query(MarkdownTemplate).all()
for template in templates:
template_dict = template.to_dict()
filename = f"{template_dict['title']}.json"
template_dict.pop('id', None)
with open(os.path.join('template', filename), 'w') as f:
json.dump(template_dict, f, default=str, indent=2)
def traverse(path_id, paths):
current_path = paths[path_id]
if path_id == 1:
create_and_cd("Root")
else:
if path_id != 1:
create_and_cd(current_path.name)
with open(".meta", "w") as meta_file:
meta_file.write(f"order: {current_path.order}\n")
with get_db() as session:
path_meta = {
"name": current_path.name,
"order": current_path.order,
"backup_id": path_id
}
if current_path.setting_id:
path_setting = session.query(PathSetting).get(current_path.setting_id)
if path_setting:
if path_setting.webhook_setting_id:
webhook_setting = session.query(WebhookSetting).get(path_setting.webhook_setting_id)
if webhook_setting:
if webhook_setting.webhook_id:
webhook = session.query(Webhook).get(webhook_setting.webhook_id)
if webhook:
path_meta["webhook_ref"] = {
"hook_url": webhook.hook_url
}
path_meta["webhook_setting"] = {
"recursive": webhook_setting.recursive,
"additional_header": webhook_setting.additional_header,
"enabled": webhook_setting.enabled,
"on_events": webhook_setting.on_events
}
if path_setting.template_setting_id:
path_template = session.query(PathTemplate).get(path_setting.template_setting_id)
if path_template:
path_meta["template_ref"] = {
"title": path_template.title
}
path_meta["template_setting"] = {
"template_id": path_setting.template_setting_id
}
with open(".json.meta", "w") as meta_file:
json.dump(path_meta, meta_file, default=str, indent=2)
mds = session.query(Markdown).filter(Markdown.path_id == path_id).all()
for md in mds:
with open(f"{md.title}.md", "w") as md_file:
md_file.write(md.content)
with open(f"{md.title}.mdmeta", "w") as meta_file:
meta_file.write(f"created_at: {md.created_at}\n")
meta_file.write(f"order: {md.order}\n")
meta_file.write(f"shortcut: {md.shortcut}\n")
md_data = {
"title": md.title,
"content": md.content,
"created_at": md.created_at,
"order": md.order,
"shortcut": md.shortcut,
"backup_id": md.id
}
if md.setting_id:
md_setting = session.query(MarkdownSetting).get(md.setting_id)
if md_setting:
settings = {}
if md_setting.template_setting_id:
template_setting = session.query(MarkdownTemplateSetting).get(md_setting.template_setting_id)
if template_setting and template_setting.template_id:
template = session.query(MarkdownTemplate).get(template_setting.template_id)
if template:
settings["template_setting"] = {
"template_id": template_setting.template_id,
"template_ref": {
"title": template.title
}
}
if md_setting.permission_setting_id:
permission_setting = session.query(MarkdownPermissionSetting).get(md_setting.permission_setting_id)
if permission_setting:
settings["permission_setting"] = permission_setting.permission
if settings:
md_data["setting"] = settings
with open(f"{md.title}.json", "w") as md_file:
json.dump(md_data, md_file, default=str, indent=2)
children = [c for c in paths.values() if c.parent_id == path_id]
for child in children:
traverse(child.id, paths)
cd_back()
if path_id != 1:
cd_back()
@backup_bp.route('/load', methods=['POST'])
@require_auth(roles=['admin'])
def load_backup():
"""
Restore data from a backup file.
This function restores data from a backup file, including:
- The tree structure (paths and markdowns)
- Templates
- Webhooks
If the backup version does not match the current version, the backup will be
automatically converted to the current version before being restored.
Request:
- file: The backup file to restore (multipart/form-data)
Returns:
A JSON object with a success message.
Response Codes:
- 200: Backup restored successfully
- 400: No file provided or invalid backup format
- 429: Another backup restore is in progress
- 500: Failed to restore backup
"""
if not backup_lock.acquire(blocking=False):
return jsonify({"error": "Another backup restore is in progress. Please try again later."}), 429
@@ -86,18 +554,32 @@ def load_backup():
with zipfile.ZipFile(zip_path, 'r') as zip_ref:
zip_ref.extractall(temp_dir)
root_dir = temp_dir
root_dir = os.path.join(temp_dir, "Root")
if not os.path.exists(root_dir):
return jsonify({"error": "Invalid backup format"}), 400
root_dir = temp_dir
success, error_response = check_and_convert_backup_version(root_dir)
if not success:
return error_response
tree_dir = os.path.join(root_dir, "tree")
template_dir = os.path.join(root_dir, "template")
if not os.path.exists(tree_dir) or not os.path.exists(template_dir):
return jsonify({"error": "Invalid backup format: missing tree or template directory"}), 400
with get_db() as session:
import_templates(template_dir, session)
webhook_mapping = import_webhooks(os.path.join(root_dir, "webhook.json"), session)
path_mapping = {}
restore_tree(root_dir, None, session, path_mapping)
restore_tree(tree_dir, None, session, path_mapping, webhook_mapping)
session.commit()
shutil.rmtree(temp_dir)
return jsonify({"success": True, "message": "Backup restored and merged successfully"})
return jsonify({"success": True, "message": "Backup restored and merged successfully"}), 200
except Exception as e:
logger.error(f"Failed to load backup: {e}")
@@ -106,26 +588,264 @@ def load_backup():
backup_lock.release()
def restore_tree(dir_path, parent_id, session, path_mapping):
def import_templates(template_dir, session):
template_mapping = {}
for filename in os.listdir(template_dir):
if filename.endswith('.json'):
file_path = os.path.join(template_dir, filename)
try:
with open(file_path, 'r') as f:
template_data = json.load(f)
title = template_data.get('title')
if not title:
title = os.path.splitext(filename)[0]
existing_template = session.query(MarkdownTemplate).filter_by(title=title).first()
if existing_template:
template_mapping[title] = existing_template.id
else:
new_template = MarkdownTemplate(
title=title,
parameters=template_data.get('parameters'),
layout=template_data.get('layout')
)
session.add(new_template)
session.flush()
template_mapping[title] = new_template.id
except Exception as e:
logger.error(f"Error importing template {filename}: {e}")
return template_mapping
def import_webhooks(webhook_file, session):
webhook_mapping = {}
if not os.path.exists(webhook_file):
logger.warning(f"Webhook file not found: {webhook_file}")
return webhook_mapping
try:
with open(webhook_file, 'r') as f:
webhook_data = json.load(f)
for webhook_entry in webhook_data:
backup_id = webhook_entry.get('backup_id')
hook_url = webhook_entry.get('hook_url')
if not hook_url:
continue
existing_webhook = session.query(Webhook).filter_by(hook_url=hook_url).first()
if existing_webhook:
webhook_id = existing_webhook.id
else:
new_webhook = Webhook(hook_url=hook_url)
session.add(new_webhook)
session.flush()
webhook_id = new_webhook.id
webhook_mapping[backup_id] = webhook_id
settings = webhook_entry.get('settings', [])
for setting_data in settings:
new_setting = WebhookSetting(
webhook_id=webhook_id,
recursive=setting_data.get('recursive', False),
additional_header=setting_data.get('additional_header'),
enabled=setting_data.get('enabled', True),
on_events=setting_data.get('on_events', 0)
)
session.add(new_setting)
except Exception as e:
logger.error(f"Error importing webhooks: {e}")
return webhook_mapping
def process_markdown_file(file_path, file_name, new_path_id, session):
try:
with open(file_path, "r", encoding="utf-8") as f:
md_data = json.load(f)
md_title = md_data.get("title")
if not md_title:
md_title = os.path.splitext(file_name)[0]
content = md_data.get("content", "")
created_at_str = md_data.get("created_at")
created_at = datetime.now()
if created_at_str:
try:
created_at = datetime.strptime(created_at_str, "%Y-%m-%d %H:%M:%S")
except ValueError:
pass
order = md_data.get("order", "")
shortcut = md_data.get("shortcut", "")
setting_id = None
template_setting_id = None
permission_setting_id = None
if "setting" in md_data:
settings = md_data.get("setting", {})
if "template_setting" in settings:
template_setting_data = settings["template_setting"]
template_title = None
if isinstance(template_setting_data, dict) and "template_ref" in template_setting_data:
template_title = template_setting_data["template_ref"].get("title")
if template_title:
existing_template = session.query(MarkdownTemplate).filter_by(title=template_title).first()
if existing_template:
template_id = existing_template.id
new_template_setting = MarkdownTemplateSetting(template_id=template_id)
session.add(new_template_setting)
session.flush()
template_setting_id = new_template_setting.id
if "permission_setting" in settings:
permission = settings["permission_setting"]
if permission:
new_permission_setting = MarkdownPermissionSetting(permission=permission)
session.add(new_permission_setting)
session.flush()
permission_setting_id = new_permission_setting.id
else:
if "template_ref" in md_data:
template_title = md_data["template_ref"].get("title")
if template_title:
existing_template = session.query(MarkdownTemplate).filter_by(title=template_title).first()
if existing_template:
template_id = existing_template.id
if "template_setting" in md_data:
pass
new_template_setting = MarkdownTemplateSetting(template_id=template_id)
session.add(new_template_setting)
session.flush()
template_setting_id = new_template_setting.id
if "permission" in md_data:
permission = md_data.get("permission")
if permission:
new_permission_setting = MarkdownPermissionSetting(permission=permission)
session.add(new_permission_setting)
session.flush()
permission_setting_id = new_permission_setting.id
if template_setting_id or permission_setting_id:
md_setting = MarkdownSetting(
template_setting_id=template_setting_id,
permission_setting_id=permission_setting_id
)
session.add(md_setting)
session.flush()
setting_id = md_setting.id
existing_md = session.query(Markdown).filter_by(path_id=new_path_id, title=md_title).first()
if existing_md:
existing_md.content = content
existing_md.created_at = created_at
existing_md.order = order
existing_md.shortcut = shortcut
existing_md.setting_id = setting_id
session.commit()
else:
new_md = Markdown(
title=md_title,
content=content,
path_id=new_path_id,
created_at=created_at,
order=order,
shortcut=shortcut,
setting_id=setting_id
)
session.add(new_md)
except Exception as e:
logger.error(f"Error processing markdown file {file_name}: {e}")
def restore_tree(dir_path, parent_id, session, path_mapping, webhook_mapping=None):
if webhook_mapping is None:
webhook_mapping = {}
dir_name = os.path.basename(dir_path)
existing_path = session.query(Path).filter_by(parent_id=parent_id, name=dir_name).first()
if parent_id is None:
if dir_name == "Root" or dir_name == "tree":
new_path_id = 1
elif existing_path:
path_mapping[dir_path] = new_path_id
for item in os.listdir(dir_path):
item_path = os.path.join(dir_path, item)
if os.path.isdir(item_path):
restore_tree(item_path, 1, session, path_mapping, webhook_mapping)
elif item.endswith(".json") and not item == ".json.meta":
process_markdown_file(item_path, item, new_path_id, session)
return
existing_path = session.query(Path).filter_by(parent_id=parent_id, name=dir_name).first()
if existing_path:
new_path_id = existing_path.id
else:
order = ''
meta_file_path = os.path.join(dir_path, ".meta")
if os.path.exists(meta_file_path):
with open(meta_file_path, "r") as meta_file:
for line in meta_file:
key, value = line.strip().split(": ", 1)
if key == "order":
order = value
setting_id = None
new_path = Path(name=dir_name, parent_id=parent_id, order=order)
meta_file_path = os.path.join(dir_path, ".json.meta")
if os.path.exists(meta_file_path):
try:
with open(meta_file_path, "r") as meta_file:
path_meta = json.load(meta_file)
order = path_meta.get("order", '')
webhook_setting_id = None
if "webhook_ref" in path_meta and "webhook_setting" in path_meta:
hook_url = path_meta["webhook_ref"].get("hook_url")
webhook_id = None
existing_webhook = session.query(Webhook).filter_by(hook_url=hook_url).first()
if existing_webhook:
webhook_id = existing_webhook.id
else:
new_webhook = Webhook(hook_url=hook_url)
session.add(new_webhook)
session.flush()
webhook_id = new_webhook.id
if webhook_id:
webhook_setting_data = path_meta["webhook_setting"]
new_webhook_setting = WebhookSetting(
webhook_id=webhook_id,
recursive=webhook_setting_data.get("recursive", False),
additional_header=webhook_setting_data.get("additional_header"),
enabled=webhook_setting_data.get("enabled", True),
on_events=webhook_setting_data.get("on_events", 0)
)
session.add(new_webhook_setting)
session.flush()
webhook_setting_id = new_webhook_setting.id
template_setting_id = None
if "template_ref" in path_meta:
template_title = path_meta["template_ref"].get("title")
existing_template = session.query(PathTemplate).filter_by(title=template_title).first()
if existing_template:
template_setting_id = existing_template.id
if webhook_setting_id or template_setting_id:
path_setting = PathSetting(
webhook_setting_id=webhook_setting_id,
template_setting_id=template_setting_id
)
session.add(path_setting)
session.flush()
setting_id = path_setting.id
except Exception as e:
logger.error(f"Error parsing path metadata: {e}")
new_path = Path(name=dir_name, parent_id=parent_id, order=order, setting_id=setting_id)
session.add(new_path)
session.flush()
new_path_id = new_path.id
@@ -134,44 +854,10 @@ def restore_tree(dir_path, parent_id, session, path_mapping):
for file in os.listdir(dir_path):
file_path = os.path.join(dir_path, file)
if file.endswith(".md"):
md_title = file[:-3]
mdmeta_path = file_path + "meta"
created_at = datetime.now()
order = ''
shortcut = ""
if os.path.exists(mdmeta_path):
with open(mdmeta_path, "r") as meta_file:
for line in meta_file:
key, value = line.strip().split(": ", 1)
if key == "created_at":
created_at = datetime.strptime(value, "%Y-%m-%d %H:%M:%S")
elif key == "order":
order = value
elif key == "shortcut":
shortcut = value
with open(file_path, "r", encoding="utf-8") as md_file:
content = md_file.read()
unique_title = get_unique_markdown_title(session, md_title, new_path_id)
new_md = Markdown(title=unique_title, content=content, path_id=new_path_id,
created_at=created_at, order=order, shortcut=shortcut)
session.add(new_md)
if file.endswith(".json") and not file == ".json.meta":
process_markdown_file(file_path, file, new_path_id, session)
for item in os.listdir(dir_path):
item_path = os.path.join(dir_path, item)
if os.path.isdir(item_path):
restore_tree(item_path, new_path_id, session, path_mapping)
def get_unique_markdown_title(session, title, path_id):
existing_titles = {md.title for md in session.query(Markdown.title).filter_by(path_id=path_id).all()}
unique_title = title
while unique_title in existing_titles:
unique_title += ".bp"
return unique_title
restore_tree(item_path, new_path_id, session, path_mapping, webhook_mapping)

33
api/config.py
View File

@@ -12,17 +12,48 @@ def is_valid_rate_limit(limit):
@require_auth(roles=['admin'])
@etag_response
def limits():
"""
Get all rate limits.
This endpoint retrieves a list of all rate limits configured in the system.
It requires authentication with the 'admin' role.
Returns:
A JSON object containing all rate limits, with keys in the format "endpoint : method".
Response Codes:
- 200: Success
"""
return jsonify(rate_limits), 200
@config_bp.route('/limits', methods=['PUT'])
@require_auth(roles=['admin'])
def update_limits():
"""
Update a rate limit.
This endpoint updates the rate limit for a specific endpoint and method.
It requires authentication with the 'admin' role.
Request:
- endpoint (str): The endpoint path to update
- method (str): The HTTP method to update
- new_limit (str): The new rate limit value (format: "number per second/minute/hour/day")
Returns:
A JSON object with a success message.
Response Codes:
- 200: Updated successfully
- 400: Bad request (missing required fields or invalid rate limit format)
- 404: Endpoint not found
"""
data = request.json
if not data or 'endpoint' not in data or 'method' not in data or 'new_limit' not in data:
return jsonify({'error': 'Bad request'}), 400
key = f"{data['endpoint']} : {data['method']}"
if key not in rate_limits:
return jsonify({'error': 'endpoint not fount'}), 404
return jsonify({'error': 'endpoint not found'}), 404
if is_valid_rate_limit(data['new_limit']):
rate_limits[key] = data['new_limit']
return jsonify({"message": "updated"}), 200

197
api/markdown.py
View File

@@ -19,6 +19,17 @@ markdown_bp = Blueprint('markdown', __name__, url_prefix='/api/markdown')
@limiter.limit(api.get_rate_limit)
@etag_response
def get_markdowns():
"""
Get all markdown documents.
This endpoint retrieves a list of all markdown documents in the system.
Returns:
A JSON array containing all markdown documents.
Response Codes:
- 200: Success
"""
with get_db() as session:
mds = session.query(Markdown).all()
return jsonify([md.to_dict() for md in mds]), 200
@@ -27,6 +38,19 @@ def get_markdowns():
@limiter.limit(api.get_rate_limit)
@etag_response
def get_home():
"""
Get the home markdown document.
This endpoint retrieves the index markdown document from the root path (path_id=1).
This is typically the main landing page content.
Returns:
A JSON object containing the home markdown document.
Response Codes:
- 200: Success
- 204: No content (home markdown not found)
"""
with get_db() as session:
markdown = session.query(Markdown).filter(Markdown.path_id == 1, Markdown.title == "index").first()
if markdown is None:
@@ -37,6 +61,20 @@ def get_home():
@limiter.limit(api.get_rate_limit)
@etag_response
def get_markdowns_by_path(path_id):
"""
Get all markdown documents in a specific path.
This endpoint retrieves a list of all markdown documents that belong to the specified path.
Request:
- path_id (int): The ID of the path to get markdowns from
Returns:
A JSON array containing all markdown documents in the specified path.
Response Codes:
- 200: Success
"""
with get_db() as session:
markdowns = session.query(Markdown).filter(Markdown.path_id == path_id).all()
return jsonify([md.to_dict() for md in markdowns]), 200
@@ -45,6 +83,22 @@ def get_markdowns_by_path(path_id):
@limiter.limit(api.get_rate_limit)
@etag_response
def get_index(path_id):
"""
Get the index markdown document for a specific path.
This endpoint retrieves the index markdown document from the specified path.
The index document typically serves as the main content or landing page for a path.
Request:
- path_id (int): The ID of the path to get the index markdown from
Returns:
A JSON object containing the index markdown document.
Response Codes:
- 200: Success
- 204: No content (index markdown not found)
"""
with get_db() as session:
markdown = session.query(Markdown).filter(Markdown.path_id == path_id).filter(Markdown.title == "index").first()
if markdown is None:
@@ -57,6 +111,24 @@ def get_index(path_id):
@limiter.limit(api.get_rate_limit)
@etag_response
def get_markdown(markdown_id):
"""
Get a specific markdown document by ID.
This endpoint retrieves a markdown document by its ID. It includes permission checks
based on the user's role and the markdown's permission settings.
Request:
- markdown_id (int): The ID of the markdown document to retrieve
Returns:
A JSON object containing the markdown document.
Response Codes:
- 200: Success
- 203: Permission denied (for protected markdowns when user is not an admin)
- 403: Permission denied (for private markdowns when user is not an admin)
- 404: Markdown not found
"""
is_admin = is_user_admin()
with get_db() as session:
@@ -80,6 +152,27 @@ def get_markdown(markdown_id):
@require_auth(roles=['admin', 'creator'])
@limiter.limit(api.get_rate_limit)
def create_markdown():
"""
Create a new markdown document.
This endpoint creates a new markdown document with the provided data.
It requires authentication with either 'admin' or 'creator' role.
Request:
- title (str): The title of the markdown document
- content (str): The content of the markdown document
- path_id (int): The ID of the path where the markdown will be created
- shortcut (str, optional): A shortcut identifier for the markdown
- setting_id (int, optional): The ID of the markdown settings
Returns:
A JSON object containing the created markdown document.
Response Codes:
- 201: Created successfully
- 400: Bad request (missing required fields or duplicate shortcut)
- 500: Server error
"""
data = request.json
title = data.get('title')
content = data.get('content')
@@ -109,6 +202,31 @@ def create_markdown():
@require_auth(roles=['admin', 'creator'])
@limiter.limit(api.get_rate_limit)
def update_markdown(markdown_id):
"""
Update a markdown document.
This endpoint updates an existing markdown document with the provided data.
It requires authentication with either 'admin' or 'creator' role.
- PUT: Replaces the entire markdown document
- PATCH: Updates only the specified fields
Request:
- markdown_id (int): The ID of the markdown document to update
- title (str, optional for PATCH): The new title for the markdown
- content (str, optional for PATCH): The new content for the markdown
- path_id (int, optional for PATCH): The new path ID for the markdown
- shortcut (str, optional): A shortcut identifier for the markdown
- setting_id (int, optional): The ID of the markdown settings
Returns:
A JSON object containing the updated markdown document.
Response Codes:
- 200: Updated successfully
- 400: Bad request (duplicate shortcut)
- 404: Markdown not found
"""
with get_db() as session:
markdown = session.query(Markdown).get(markdown_id)
if markdown is None:
@@ -147,6 +265,22 @@ def update_markdown(markdown_id):
@require_auth(roles=['admin'])
@limiter.limit(api.get_rate_limit)
def delete_markdown(markdown_id):
"""
Delete a markdown document.
This endpoint deletes an existing markdown document.
It requires authentication with the 'admin' role.
Request:
- markdown_id (int): The ID of the markdown document to delete
Returns:
A JSON object containing the deleted markdown document.
Response Codes:
- 200: Deleted successfully
- 404: Markdown not found
"""
with get_db() as session:
markdown = session.get(Markdown, markdown_id)
if markdown is None:
@@ -164,6 +298,24 @@ def delete_markdown(markdown_id):
@require_auth(roles=['admin'])
@limiter.limit(api.get_rate_limit)
def move_forward(markdown_id):
"""
Move a markdown document forward in display order.
This endpoint moves a markdown document one position forward in the display order by swapping its order value
with the previous markdown in the same path. This affects how markdowns are displayed in the UI.
It requires authentication with the 'admin' role.
Request:
- markdown_id (int): The ID of the markdown document to move forward
Returns:
A JSON object containing the updated markdown document.
Response Codes:
- 200: Moved successfully
- 400: Bad request (already at the first position)
- 404: Markdown not found
"""
with get_db() as session:
markdown = session.query(Markdown).get(markdown_id)
if not markdown:
@@ -183,6 +335,24 @@ def move_forward(markdown_id):
@require_auth(roles=['admin'])
@limiter.limit(api.get_rate_limit)
def move_backward(markdown_id):
"""
Move a markdown document backward in display order.
This endpoint moves a markdown document one position backward in the display order by swapping its order value
with the next markdown in the same path. This affects how markdowns are displayed in the UI.
It requires authentication with the 'admin' role.
Request:
- markdown_id (int): The ID of the markdown document to move backward
Returns:
A JSON object containing the updated markdown document.
Response Codes:
- 200: Moved successfully
- 400: Bad request (already at the last position)
- 404: Markdown not found
"""
with get_db() as session:
markdown = session.get(Markdown, markdown_id)
if not markdown:
@@ -201,6 +371,21 @@ def move_backward(markdown_id):
@markdown_bp.route('/search/<string:keyword>', methods=['GET'])
@limiter.limit(api.get_rate_limit)
def search_markdowns(keyword):
"""
Search for markdown documents.
This endpoint searches for markdown documents that contain the specified keyword
in either their title or content.
Request:
- keyword (str): The search term to look for in markdown titles and content
Returns:
A JSON array containing all markdown documents that match the search criteria.
Response Codes:
- 200: Success
"""
with get_db() as session:
res = session.query(Markdown).filter(
or_(Markdown.title.like(keyword), Markdown.content.like(keyword))
@@ -210,6 +395,18 @@ def search_markdowns(keyword):
@markdown_bp.route('/links', methods=['GET'])
@limiter.limit(api.get_rate_limit)
def get_links():
"""
Get all markdown shortcut links.
This endpoint retrieves a list of all markdown documents that have a shortcut defined,
formatted as links in the format "[shortcut]: id".
Returns:
A JSON array containing formatted links for all markdown documents with shortcuts.
Response Codes:
- 200: Success
"""
with get_db() as session:
mds = [md.to_dict() for md in session.query(Markdown).filter(Markdown.shortcut != "").all()]
links = [f"[{md['shortcut']}]: {md['id']}" for md in mds]

155
api/path.py
View File

@@ -18,6 +18,17 @@ path_bp = Blueprint('path', __name__, url_prefix='/api/path')
@limiter.limit(api.get_rate_limit)
@etag_response
def get_root_paths():
"""
Get all paths under the root path.
This endpoint retrieves a list of all paths that are direct children of the root path (parent_id=1).
Returns:
A JSON array containing all root paths.
Response Codes:
- 200: Success
"""
with get_db() as session:
paths = session.query(Path).filter(Path.parent_id == 1)
return jsonify([pth.to_dict() for pth in paths]), 200
@@ -26,6 +37,21 @@ def get_root_paths():
@limiter.limit(api.get_rate_limit)
@etag_response
def get_path(path_id):
"""
Get a specific path by ID.
This endpoint retrieves a path by its ID.
Request:
- path_id (int): The ID of the path to retrieve
Returns:
A JSON object containing the path.
Response Codes:
- 200: Success
- 404: Path not found
"""
with get_db() as session:
path = session.query(Path).get(path_id)
if path is None:
@@ -36,6 +62,20 @@ def get_path(path_id):
@limiter.limit(api.get_rate_limit)
@etag_response
def get_path_by_parent(parent_id):
"""
Get all paths under a specific parent path.
This endpoint retrieves a list of all paths that are direct children of the specified parent path.
Request:
- parent_id (int): The ID of the parent path
Returns:
A JSON array containing all child paths of the specified parent.
Response Codes:
- 200: Success
"""
with get_db() as session:
paths = session.query(Path).filter(Path.parent_id == parent_id).all()
return jsonify([pth.to_dict() for pth in paths]), 200
@@ -44,6 +84,25 @@ def get_path_by_parent(parent_id):
@limiter.limit(api.get_rate_limit)
@require_auth(roles=['admin', 'creator'])
def create_path():
"""
Create a new path.
This endpoint creates a new path with the provided name and parent ID.
It requires authentication with either 'admin' or 'creator' role.
Request:
- name (str): The name of the path
- parent_id (int): The ID of the parent path
Returns:
A JSON object containing the created path.
Response Codes:
- 201: Created successfully
- 400: Bad request (missing required fields)
- 404: Parent path not found
- 409: Conflict (path already exists under the parent)
"""
data = request.json
if not data or 'name' not in data or 'parent_id' not in data:
return jsonify({"error": "bad request"}), 400
@@ -62,6 +121,26 @@ def create_path():
@limiter.limit(api.get_rate_limit)
@require_auth(roles=['admin'])
def update_path(path_id):
"""
Update a path.
This endpoint updates an existing path with the provided name and parent ID.
It requires authentication with the 'admin' role.
Request:
- path_id (int): The ID of the path to update
- name (str): The new name for the path
- parent_id (int): The new parent ID for the path
Returns:
A JSON object containing the updated path.
Response Codes:
- 200: Updated successfully
- 400: Bad request (missing required fields)
- 404: Path not found
- 409: Conflict (path already exists under the parent)
"""
data = request.json
if not data or 'name' not in data or 'parent_id' not in data:
return jsonify({"error": "bad request"}), 400
@@ -81,6 +160,28 @@ def update_path(path_id):
@limiter.limit(api.get_rate_limit)
@require_auth(roles=['admin'])
def patch_path(path_id):
"""
Partially update a path.
This endpoint partially updates an existing path with the provided data.
Unlike the PUT method, this endpoint only updates the fields that are provided in the request.
It requires authentication with the 'admin' role.
Request:
- path_id (int): The ID of the path to update
- name (str, optional): The new name for the path
- parent_id (int, optional): The new parent ID for the path
- setting_id (int, optional): The new setting ID for the path
Returns:
A JSON object containing the updated path.
Response Codes:
- 200: Updated successfully
- 400: Bad request (empty data)
- 404: Path not found
- 409: Conflict (path already exists under the parent)
"""
data = request.json
if not data:
return jsonify({"error": "bad request"}), 400
@@ -106,6 +207,23 @@ def patch_path(path_id):
@limiter.limit(api.get_rate_limit)
@require_auth(roles=['admin'])
def delete_path(path_id):
"""
Delete a path.
This endpoint deletes an existing path. The path must be empty (no child paths or markdowns)
to be deleted. It requires authentication with the 'admin' role.
Request:
- path_id (int): The ID of the path to delete
Returns:
A JSON object with a success message.
Response Codes:
- 200: Deleted successfully
- 404: Path not found
- 409: Conflict (path contains child paths or markdowns)
"""
with get_db() as session:
path = session.query(Path).get(path_id)
if not path:
@@ -125,6 +243,24 @@ def delete_path(path_id):
@require_auth(roles=['admin'])
@limiter.limit(api.get_rate_limit)
def move_forward(path_id):
"""
Move a path forward in display order.
This endpoint moves a path one position forward in the display order by swapping its order value
with the previous path in the same parent. This affects how paths are displayed in the UI.
It requires authentication with the 'admin' role.
Request:
- path_id (int): The ID of the path to move forward
Returns:
A JSON object containing the updated path.
Response Codes:
- 200: Moved successfully
- 400: Bad request (already at the first position)
- 404: Path not found
"""
with get_db() as session:
path = session.query(Path).get(path_id)
if not path:
@@ -144,6 +280,24 @@ def move_forward(path_id):
@require_auth(roles=['admin'])
@limiter.limit(api.get_rate_limit)
def move_backward(path_id):
"""
Move a path backward in display order.
This endpoint moves a path one position backward in the display order by swapping its order value
with the next path in the same parent. This affects how paths are displayed in the UI.
It requires authentication with the 'admin' role.
Request:
- path_id (int): The ID of the path to move backward
Returns:
A JSON object containing the updated path.
Response Codes:
- 200: Moved successfully
- 400: Bad request (already at the last position)
- 404: Path not found
"""
with get_db() as session:
path = session.query(Path).get(path_id)
if not path:
@@ -158,4 +312,3 @@ def move_backward(path_id):
session.commit()
path_updated.send(None, payload=path.to_dict())
return jsonify(path.to_dict()), 200

67
api/setting/markdown/__init__.py
View File

@@ -11,6 +11,21 @@ from db.models.MarkdownSetting import MarkdownSetting
@limiter.limit(api.get_rate_limit)
@etag_response
def get_markdown_path(setting_id):
"""
Get a specific markdown setting by ID.
This endpoint retrieves a markdown setting by its ID.
Request:
- setting_id (int): The ID of the markdown setting to retrieve
Returns:
A JSON object containing the markdown setting.
Response Codes:
- 200: Success
- 204: No content (setting not found)
"""
with get_db() as session:
setting = session.query(MarkdownSetting).get(setting_id)
if setting is None:
@@ -21,6 +36,23 @@ def get_markdown_path(setting_id):
@setting_bp.route('/markdown/', methods=['POST'])
@require_auth(roles=['admin'])
def create_markdown_setting():
"""
Create a new markdown setting.
This endpoint creates a new markdown setting with the provided template and permission settings.
It requires authentication with the 'admin' role.
Request:
- template_setting_id (int, optional): The ID of the template setting to associate
- permission_setting_id (int, optional): The ID of the permission setting to associate
Returns:
A JSON object containing the created markdown setting.
Response Codes:
- 200: Created successfully
- 500: Server error
"""
data = request.json
template_setting_id = data.get('template_setting_id')
permission_setting_id = data.get('permission_setting_id')
@@ -39,6 +71,25 @@ def create_markdown_setting():
@setting_bp.route('/markdown/<int:setting_id>', methods=['PUT', 'PATCH'])
@require_auth(roles=['admin'])
def update_markdown_setting(setting_id):
"""
Update a markdown setting.
This endpoint updates an existing markdown setting with the provided template and permission settings.
It requires authentication with the 'admin' role.
Request:
- setting_id (int): The ID of the markdown setting to update
- template_setting_id (int, optional): The new template setting ID
- permission_setting_id (int, optional): The new permission setting ID
Returns:
A JSON object containing the updated markdown setting.
Response Codes:
- 200: Updated successfully
- 400: Bad request (setting not found)
- 500: Server error
"""
data = request.json
try:
with get_db() as session:
@@ -58,6 +109,22 @@ def update_markdown_setting(setting_id):
@setting_bp.route('/markdown/<int:setting_id>', methods=['DELETE'])
@require_auth(roles=['admin'])
def delete_markdown_setting(setting_id):
"""
Delete a markdown setting.
This endpoint deletes an existing markdown setting.
It requires authentication with the 'admin' role.
Request:
- setting_id (int): The ID of the markdown setting to delete
Returns:
A JSON object with a success message.
Response Codes:
- 200: Deleted successfully
- 400: Bad request (setting not found)
"""
with get_db() as session:
setting = session.query(MarkdownSetting).get(setting_id)
if setting is None:

65
api/setting/markdown/permission.py
View File

@@ -10,6 +10,21 @@ from db.models.MarkdownPermissionSetting import MarkdownPermissionSetting
@etag_response
@limiter.limit(api.get_rate_limit)
def get_permission_setting(setting_id):
"""
Get a specific markdown permission setting by ID.
This endpoint retrieves a markdown permission setting by its ID.
Request:
- setting_id (int): The ID of the permission setting to retrieve
Returns:
A JSON object containing the permission setting.
Response Codes:
- 200: Success
- 204: No content (setting not found)
"""
with get_db() as session:
setting = session.query(MarkdownPermissionSetting).get(setting_id)
if not setting:
@@ -20,6 +35,21 @@ def get_permission_setting(setting_id):
@setting_bp.route('/markdown/permission/', methods=['POST'])
@require_auth(roles=['admin'])
def create_permission_setting():
"""
Create a new markdown permission setting.
This endpoint creates a new markdown permission setting with the provided permission value.
It requires authentication with the 'admin' role.
Request:
- permission (str): The permission value (e.g., 'private', 'protected', etc.)
Returns:
A JSON object containing the created permission setting.
Response Codes:
- 201: Created successfully
"""
data = request.json
permission = data.get('permission')
new_setting = MarkdownPermissionSetting(permission=permission)
@@ -32,6 +62,26 @@ def create_permission_setting():
@setting_bp.route('/markdown/permission/<int:setting_id>', methods=['PUT', 'PATCH'])
@require_auth(roles=['admin'])
def update_permission_setting(setting_id):
"""
Update a markdown permission setting.
This endpoint updates an existing markdown permission setting with the provided permission value.
It requires authentication with the 'admin' role.
- PUT: Replaces the entire permission setting
- PATCH: Updates only the specified fields
Request:
- setting_id (int): The ID of the permission setting to update
- permission (str): The new permission value
Returns:
A JSON object containing the updated permission setting.
Response Codes:
- 200: Updated successfully
- 404: Permission setting not found
"""
with get_db() as session:
setting = session.get(MarkdownPermissionSetting, setting_id)
if setting is None:
@@ -48,6 +98,21 @@ def update_permission_setting(setting_id):
@setting_bp.route('/markdown/permission/<int:setting_id>', methods=['DELETE'])
@require_auth(roles=['admin'])
def delete_permission_setting(setting_id):
"""
Delete a markdown permission setting.
This endpoint deletes an existing markdown permission setting.
It requires authentication with the 'admin' role.
Request:
- setting_id (int): The ID of the permission setting to delete
Returns:
A JSON object containing the deleted permission setting.
Response Codes:
- 200: Deleted successfully
"""
with get_db() as session:
setting = session.get(MarkdownPermissionSetting, setting_id)
st = setting.to_dict()

76
api/setting/markdown/template.py
View File

@@ -9,6 +9,17 @@ from db.models.MarkdownTemplateSetting import MarkdownTemplateSetting
@setting_bp.route('/markdown/template/', methods=['GET'])
@etag_response
def list_template_settings():
"""
List all markdown template settings.
This endpoint retrieves a list of all markdown template settings in the system.
Returns:
A JSON array containing all template settings.
Response Codes:
- 200: Success
"""
with get_db() as session:
settings = session.query(MarkdownTemplateSetting).all()
return jsonify([s.to_dict() for s in settings]), 200
@@ -17,6 +28,21 @@ def list_template_settings():
@etag_response
@limiter.limit(api.get_rate_limit)
def get_template_setting(setting_id):
"""
Get a specific markdown template setting by ID.
This endpoint retrieves a markdown template setting by its ID.
Request:
- setting_id (int): The ID of the template setting to retrieve
Returns:
A JSON object containing the template setting.
Response Codes:
- 200: Success
- 204: No content (setting not found)
"""
with get_db() as session:
setting = session.query(MarkdownTemplateSetting).get(setting_id)
if not setting:
@@ -27,6 +53,21 @@ def get_template_setting(setting_id):
@setting_bp.route('/markdown/template/', methods=['POST'])
@require_auth(roles=['admin'])
def create_template_setting():
"""
Create a new markdown template setting.
This endpoint creates a new markdown template setting with the provided template ID.
It requires authentication with the 'admin' role.
Request:
- template_id (int): The ID of the template to associate with this setting
Returns:
A JSON object containing the created template setting.
Response Codes:
- 201: Created successfully
"""
data = request.json
template_id = data.get('template_id')
new_setting = MarkdownTemplateSetting(template_id=template_id)
@@ -39,6 +80,26 @@ def create_template_setting():
@setting_bp.route('/markdown/template/<int:setting_id>', methods=['PUT', 'PATCH'])
@require_auth(roles=['admin'])
def update_template_setting(setting_id):
"""
Update a markdown template setting.
This endpoint updates an existing markdown template setting with the provided template ID.
It requires authentication with the 'admin' role.
- PUT: Replaces the entire template setting
- PATCH: Updates only the specified fields
Request:
- setting_id (int): The ID of the template setting to update
- template_id (int): The new template ID
Returns:
A JSON object containing the updated template setting.
Response Codes:
- 200: Updated successfully
- 404: Template setting not found
"""
with get_db() as session:
setting = session.get(MarkdownTemplateSetting, setting_id)
if setting is None:
@@ -55,6 +116,21 @@ def update_template_setting(setting_id):
@setting_bp.route('/markdown/template/<int:setting_id>', methods=['DELETE'])
@require_auth(roles=['admin'])
def delete_template_setting(setting_id):
"""
Delete a markdown template setting.
This endpoint deletes an existing markdown template setting.
It requires authentication with the 'admin' role.
Request:
- setting_id (int): The ID of the template setting to delete
Returns:
A JSON object containing the deleted template setting.
Response Codes:
- 200: Deleted successfully
"""
with get_db() as session:
setting = session.get(MarkdownTemplateSetting, setting_id)
st = setting.to_dict()

68
api/setting/path/__init__.py
View File

@@ -13,6 +13,21 @@ logger = logging.getLogger(__name__)
@limiter.limit(api.get_rate_limit)
@etag_response
def get_path_setting(setting_id):
"""
Get a specific path setting by ID.
This endpoint retrieves a path setting by its ID.
Request:
- setting_id (int): The ID of the path setting to retrieve
Returns:
A JSON object containing the path setting.
Response Codes:
- 200: Success
- 204: No content (setting not found)
"""
with get_db() as session:
setting = session.query(PathSetting).get(setting_id)
if setting is None:
@@ -22,6 +37,23 @@ def get_path_setting(setting_id):
@setting_bp.route('/path/', methods=['POST'])
@require_auth(roles=['admin'])
def create_path_setting():
"""
Create a new path setting.
This endpoint creates a new path setting with the provided webhook and template settings.
It requires authentication with the 'admin' role.
Request:
- webhook_setting_id (int, optional): The ID of the webhook setting to associate
- template_setting_id (int, optional): The ID of the template setting to associate
Returns:
A JSON object containing the created path setting.
Response Codes:
- 200: Created successfully
- 500: Server error
"""
data = request.json
webhook_setting_id = data.get('webhook_setting_id')
template_setting_id = data.get('template_setting_id')
@@ -42,6 +74,25 @@ def create_path_setting():
@setting_bp.route('/path/<int:setting_id>', methods=['PUT', 'PATCH'])
@require_auth(roles=['admin'])
def update_path_setting(setting_id):
"""
Update a path setting.
This endpoint updates an existing path setting with the provided webhook and template settings.
It requires authentication with the 'admin' role.
Request:
- setting_id (int): The ID of the path setting to update
- webhook_setting_id (int, optional): The new webhook setting ID
- template_setting_id (int, optional): The new template setting ID
Returns:
A JSON object containing the updated path setting.
Response Codes:
- 200: Updated successfully
- 400: Bad request (setting not found)
- 500: Server error
"""
data = request.json
with get_db() as session:
try:
@@ -60,10 +111,25 @@ def update_path_setting(setting_id):
@setting_bp.route('/path/<int:setting_id>', methods=['DELETE'])
@require_auth(roles=['admin'])
def delete_path_setting(setting_id):
"""
Delete a path setting.
This endpoint deletes an existing path setting.
It requires authentication with the 'admin' role.
Request:
- setting_id (int): The ID of the path setting to delete
Returns:
A JSON object with a success message.
Response Codes:
- 200: Deleted successfully
- 400: Bad request (setting not found)
"""
with get_db() as session:
setting = session.query(PathSetting).get(setting_id)
if setting is None:
return jsonify({"error": "setting not exists"}), 400
session.delete(setting)
return jsonify({"message": "deleted"}), 200

85
api/setting/path/webhook.py
View File

@@ -8,6 +8,18 @@ from db.models.WebhookSetting import WebhookSetting
@setting_bp.route('/path/webhook/', methods=['GET'])
@require_auth(roles=['admin'])
def list_webhook_settings():
"""
List all webhook settings.
This endpoint retrieves a list of all webhook settings in the system.
It requires authentication with the 'admin' role.
Returns:
A JSON array containing all webhook settings.
Response Codes:
- 200: Success
"""
with get_db() as session:
settings = session.query(WebhookSetting).all()
return jsonify([s.to_dict() for s in settings]), 200
@@ -16,6 +28,22 @@ def list_webhook_settings():
@setting_bp.route('/path/webhook/<int:setting_id>', methods=['GET'])
@require_auth(roles=['admin'])
def webhook_setting(setting_id):
"""
Get a specific webhook setting by ID.
This endpoint retrieves a webhook setting by its ID.
It requires authentication with the 'admin' role.
Request:
- setting_id (int): The ID of the webhook setting to retrieve
Returns:
A JSON object containing the webhook setting.
Response Codes:
- 200: Success
- 204: No content (setting not found)
"""
with get_db() as session:
setting = session.query(WebhookSetting).filter(WebhookSetting.id == setting_id).first()
if not setting:
@@ -26,6 +54,25 @@ def webhook_setting(setting_id):
@setting_bp.route('/path/webhook/', methods=['POST'])
@require_auth(roles=['admin'])
def create_webhook_setting():
"""
Create a new webhook setting.
This endpoint creates a new webhook setting with the provided parameters.
It requires authentication with the 'admin' role.
Request:
- webhook_id (int): The ID of the webhook to associate
- recursive (bool, optional): Whether the webhook applies to subpaths (default: false)
- additional_header (str, optional): Additional headers for the webhook (default: '')
- enabled (bool, optional): Whether the webhook is enabled (default: true)
- on_events (int, optional): Event types that trigger the webhook (default: 1)
Returns:
A JSON object containing the created webhook setting.
Response Codes:
- 201: Created successfully
"""
data = request.json
with get_db() as session:
setting = WebhookSetting(
@@ -43,6 +90,27 @@ def create_webhook_setting():
@setting_bp.route('/path/webhook/<int:setting_id>', methods=['PUT', 'PATCH'])
@require_auth(roles=['admin'])
def update_webhook_setting(setting_id):
"""
Update a webhook setting.
This endpoint updates an existing webhook setting with the provided parameters.
It requires authentication with the 'admin' role.
Request:
- setting_id (int): The ID of the webhook setting to update
- webhook_id (int, optional): The new webhook ID
- recursive (bool, optional): Whether the webhook applies to subpaths
- additional_header (str, optional): Additional headers for the webhook
- enabled (bool, optional): Whether the webhook is enabled
- on_events (int, optional): Event types that trigger the webhook
Returns:
A JSON object containing the updated webhook setting.
Response Codes:
- 200: Updated successfully
- 404: Webhook setting not found
"""
data = request.json
with get_db() as session:
setting = session.query(WebhookSetting).get(setting_id)
@@ -66,6 +134,22 @@ def update_webhook_setting(setting_id):
@setting_bp.route('/path/webhook/<int:setting_id>', methods=['DELETE'])
@require_auth(roles=['admin'])
def delete_webhook_setting(setting_id):
"""
Delete a webhook setting.
This endpoint deletes an existing webhook setting.
It requires authentication with the 'admin' role.
Request:
- setting_id (int): The ID of the webhook setting to delete
Returns:
A JSON object with a success message.
Response Codes:
- 200: Deleted successfully
- 404: Webhook setting not found
"""
with get_db() as session:
setting = session.query(WebhookSetting).get(setting_id)
if not setting:
@@ -74,4 +158,3 @@ def delete_webhook_setting(setting_id):
session.delete(setting)
session.commit()
return jsonify({'message': 'Webhook setting deleted'}), 200

82
api/template/markdown/__init__.py
View File

@@ -22,6 +22,22 @@ def inflate_template(template):
@template_bp.route('/markdown/<int:template_id>', methods=['GET'])
@etag_response
def get_markdown_template(template_id):
"""
Get a specific markdown template by ID.
This endpoint retrieves a markdown template by its ID.
The template is inflated to include any nested templates.
Request:
- template_id (int): The ID of the template to retrieve
Returns:
A JSON object containing the markdown template.
Response Codes:
- 200: Success
- 204: No content (template not found)
"""
with get_db() as session:
template = session.query(MarkdownTemplate).get(template_id)
if template is None:
@@ -32,6 +48,18 @@ def get_markdown_template(template_id):
@template_bp.route('/markdown/', methods=['GET'])
@etag_response
def get_markdown_templates():
"""
List all markdown templates.
This endpoint retrieves a list of all markdown templates in the system.
Each template is inflated to include any nested templates.
Returns:
A JSON array containing all markdown templates.
Response Codes:
- 200: Success
"""
with get_db() as session:
templates = session.query(MarkdownTemplate).all()
print(templates)
@@ -41,6 +69,24 @@ def get_markdown_templates():
@template_bp.route('/markdown/', methods=['POST'])
@require_auth(roles=['admin'])
def create_markdown_template():
"""
Create a new markdown template.
This endpoint creates a new markdown template with the provided data.
It requires authentication with the 'admin' role.
Request:
- title (str): The title of the template (required)
- parameters (object, optional): The parameters for the template
- layout (str, optional): The layout content of the template
Returns:
A JSON object containing the created template.
Response Codes:
- 200: Created successfully
- 400: Bad request (missing title or other error)
"""
data = request.json
if "title" not in data:
return jsonify({"error": "title is missing"}), 400
@@ -60,6 +106,25 @@ def create_markdown_template():
@template_bp.route('/markdown/<int:template_id>', methods=['PUT', 'PATCH'])
@require_auth(roles=['admin'])
def update_markdown_template(template_id):
"""
Update a markdown template.
This endpoint updates an existing markdown template with the provided data.
It requires authentication with the 'admin' role.
Request:
- template_id (int): The ID of the template to update
- title (str, optional): The new title for the template
- parameters (object, optional): The new parameters for the template
- layout (str, optional): The new layout content for the template
Returns:
A JSON object containing the updated template.
Response Codes:
- 200: Updated successfully
- 400: Bad request (template not found)
"""
data = request.json
with get_db() as session:
template = session.query(MarkdownTemplate).get(template_id)
@@ -80,6 +145,22 @@ def update_markdown_template(template_id):
@template_bp.route('/markdown/<int:template_id>', methods=['DELETE'])
@require_auth(roles=['admin'])
def delete_markdown_template(template_id):
"""
Delete a markdown template.
This endpoint deletes an existing markdown template.
It requires authentication with the 'admin' role.
Request:
- template_id (int): The ID of the template to delete
Returns:
A JSON object with a success message.
Response Codes:
- 200: Deleted successfully
- 400: Bad request (template not found)
"""
with get_db() as session:
template = session.query(MarkdownTemplate).get(template_id)
if template is None:
@@ -89,4 +170,3 @@ def delete_markdown_template(template_id):
if template_id in cached_templates.keys():
cached_templates.pop(template_id)
return jsonify({'message': 'deleted'}), 200

77
api/template/path/__init__.py
View File

@@ -9,6 +9,21 @@ from db.models.PathTemplate import PathTemplate
@template_bp.route('/path/<int:template_id>', methods=['GET'])
@etag_response
def get_path_template(template_id):
"""
Get a specific path template by ID.
This endpoint retrieves a path template by its ID.
Request:
- template_id (int): The ID of the template to retrieve
Returns:
A JSON object containing the path template.
Response Codes:
- 200: Success
- 204: No content (template not found)
"""
with get_db() as session:
template = session.query(PathTemplate).get(template_id)
if template is None:
@@ -18,6 +33,17 @@ def get_path_template(template_id):
@template_bp.route('/path/', methods=['GET'])
@etag_response
def get_path_templates():
"""
List all path templates.
This endpoint retrieves a list of all path templates in the system.
Returns:
A JSON array containing all path templates.
Response Codes:
- 200: Success
"""
with get_db() as session:
templates = session.query(PathTemplate).all()
return jsonify([template.to_dict() for template in templates]), 200
@@ -26,6 +52,23 @@ def get_path_templates():
@template_bp.route('/path/', methods=['POST'])
@require_auth(roles=['admin'])
def create_path_template():
"""
Create a new path template.
This endpoint creates a new path template with the provided data.
It requires authentication with the 'admin' role.
Request:
- title (str): The title of the template (required)
- structure (str): The structure definition for the template
Returns:
A JSON object containing the created template.
Response Codes:
- 200: Created successfully
- 400: Bad request (missing title or other error)
"""
data = request.json
if "title" not in data:
return jsonify({"error": "title is missing"}), 400
@@ -44,6 +87,24 @@ def create_path_template():
@template_bp.route('/path/<int:template_id>', methods=['PUT', 'PATCH'])
@require_auth(roles=['admin'])
def update_path_template(template_id):
"""
Update a path template.
This endpoint updates an existing path template with the provided data.
It requires authentication with the 'admin' role.
Request:
- template_id (int): The ID of the template to update
- title (str, optional): The new title for the template
- structure (str, optional): The new structure definition for the template
Returns:
A JSON object containing the updated template.
Response Codes:
- 200: Updated successfully
- 400: Bad request (template not found)
"""
data = request.json
with get_db() as session:
template = session.query(PathTemplate).get(template_id)
@@ -60,6 +121,22 @@ def update_path_template(template_id):
@template_bp.route('/path/<int:template_id>', methods=['DELETE'])
@require_auth(roles=['admin'])
def delete_path_template(template_id):
"""
Delete a path template.
This endpoint deletes an existing path template.
It requires authentication with the 'admin' role.
Request:
- template_id (int): The ID of the template to delete
Returns:
A JSON object with a success message.
Response Codes:
- 200: Deleted successfully
- 400: Bad request (template not found)
"""
with get_db() as session:
template = session.query(PathTemplate).get(template_id)
if template is None:

46
api/tree.py
View File

@@ -1,16 +1,15 @@
from flask import Blueprint, request, jsonify
from flask import Blueprint, jsonify
from sqlalchemy.orm import Session
from sqlalchemy import and_, or_
from sqlalchemy import or_
import api
from api import etag_response, verify_token, is_user_admin
from api import etag_response, is_user_admin
from db import get_db
from db.models.Markdown import Markdown
from db.models.Path import Path
from db.models.MarkdownSetting import MarkdownSetting
from db.models.MarkdownPermissionSetting import MarkdownPermissionSetting
from api import limiter
import env_provider
import logging
logger = logging.getLogger(__name__)
@@ -67,6 +66,45 @@ def build_tree(db: Session, parent_id: int = None, is_admin=False):
@limiter.limit(api.get_rate_limit)
@etag_response
def get_tree():
"""
Get the complete tree structure of paths and markdowns.
This endpoint retrieves the hierarchical tree structure of all paths and markdowns.
For non-admin users, markdowns with 'private' permission settings are filtered out.
Returns:
A JSON object representing the tree structure with the following format:
{
"id": 1,
"name": "",
"parent_id": null,
"order": "...",
"setting_id": null,
"type": "path",
"index": true/false,
"children": [
{
"id": ...,
"title": "...",
"order": "...",
"setting_id": null,
"type": "markdown"
},
{
"id": ...,
"name": "...",
"parent_id": 1,
"order": "...",
"setting_id": null,
"type": "path",
"children": [...]
}
]
}
Response Codes:
- 200: Success
"""
is_admin = is_user_admin()
with get_db() as session:

66
api/webhook.py
View File

@@ -9,6 +9,18 @@ webhook_bp = Blueprint('webhook', __name__, url_prefix='/api/webhook')
@webhook_bp.route('/', methods=['GET'])
@require_auth(roles=['admin'])
def list_webhooks():
"""
List all webhooks.
This endpoint retrieves a list of all webhooks in the system.
It requires authentication with the 'admin' role.
Returns:
A JSON array containing all webhooks.
Response Codes:
- 200: Success
"""
with get_db() as session:
hooks = session.query(Webhook).all()
return jsonify([h.to_dict() for h in hooks]), 200
@@ -17,6 +29,23 @@ def list_webhooks():
@webhook_bp.route('/', methods=['POST'])
@require_auth(['admin'])
def create_webhook():
"""
Create a new webhook.
This endpoint creates a new webhook with the provided URL.
It requires authentication with the 'admin' role.
Request:
- hook_url (str): The URL of the webhook
Returns:
A JSON object containing the created webhook.
Response Codes:
- 201: Created successfully
- 400: Bad request (hook_url is missing)
- 409: Conflict (webhook URL already exists)
"""
data = request.json
hook_url = data.get('hook_url')
if not hook_url:
@@ -35,6 +64,25 @@ def create_webhook():
@webhook_bp.route('/<int:webhook_id>', methods=['PUT', 'PATCH'])
@require_auth(roles=['admin'])
def update_webhook(webhook_id):
"""
Update a webhook.
This endpoint updates an existing webhook with the provided URL.
It requires authentication with the 'admin' role.
Request:
- webhook_id (int): The ID of the webhook to update
- hook_url (str): The new URL for the webhook
Returns:
A JSON object containing the updated webhook.
Response Codes:
- 200: Updated successfully
- 400: Bad request (hook_url is missing)
- 404: Webhook not found
- 409: Conflict (webhook URL already exists)
"""
data = request.json
if 'hook_url' not in data:
return jsonify({'error': 'hook_url is required'}), 400
@@ -56,6 +104,22 @@ def update_webhook(webhook_id):
@webhook_bp.route('/<int:webhook_id>', methods=['DELETE'])
@require_auth(roles=['admin'])
def delete_webhook(webhook_id):
"""
Delete a webhook.
This endpoint deletes an existing webhook.
It requires authentication with the 'admin' role.
Request:
- webhook_id (int): The ID of the webhook to delete
Returns:
A JSON object with a success message.
Response Codes:
- 200: Deleted successfully
- 404: Webhook not found
"""
with get_db() as session:
webhook = session.query(Webhook).get(webhook_id)
if not webhook:
@@ -63,4 +127,4 @@ def delete_webhook(webhook_id):
session.delete(webhook)
session.commit()
return jsonify({'message': 'Webhook deleted'}), 200
return jsonify({'message': 'Webhook deleted'}), 200

172
misc/backup_converters/__init__.py Normal file
View File

@@ -0,0 +1,172 @@
"""
Backup Conversion System
========================
This module provides functionality for converting backups between different versions.
Each version converter is implemented in a separate file and registered with the system.
Usage:
------
To convert a backup from one version to another:
```python
from misc.backup_converters import convert_backup
# Convert a backup from version 1.0 to the latest version
convert_backup(backup_dir, target_version=None) # None means latest version
```
Adding a New Converter:
----------------------
1. Create a new file in the backup_converters directory, e.g., v1_to_v2.py
2. Implement a function that takes a backup directory and converts it to the target version
3. Register the converter in this file using the register_converter function
Example:
```python
# In v1_to_v2.py
from misc.backup_converters import register_converter
def convert_v1_to_v2(backup_dir):
# Conversion logic here
return new_version
register_converter("1.0", "2.0", convert_v1_to_v2)
```
"""
import os
import json
import logging
from typing import Dict, Callable, Tuple, Optional, List
logger = logging.getLogger(__name__)
CURRENT_VERSION = "1.0"
_converters: Dict[Tuple[str, str], Callable[[str], str]] = {}
def register_converter(from_version: str, to_version: str, converter: Callable[[str], str]):
"""
Register a converter function for a specific version transition.
Args:
from_version (str): The source version
to_version (str): The target version
converter (callable): A function that takes a backup directory path and returns the new version
"""
_converters[(from_version, to_version)] = converter
logger.info(f"Registered converter from version {from_version} to {to_version}")
def get_backup_version(backup_dir: str) -> str:
"""
Get the version of a backup.
Args:
backup_dir (str): Path to the backup directory
Returns:
str: The version of the backup, or "0.0" if no version is found
"""
version_file = os.path.join(backup_dir, "version.json")
if os.path.exists(version_file):
try:
with open(version_file, 'r') as f:
version_data = json.load(f)
return version_data.get("version", "0.0")
except Exception as e:
logger.error(f"Error reading version file: {e}")
return "0.0"
def set_backup_version(backup_dir: str, version: str):
"""
Set the version of a backup.
Args:
backup_dir (str): Path to the backup directory
version (str): The version to set
"""
version_file = os.path.join(backup_dir, "version.json")
try:
with open(version_file, 'w') as f:
json.dump({"version": version}, f)
except Exception as e:
logger.error(f"Error writing version file: {e}")
def find_conversion_path(from_version: str, to_version: str) -> Optional[List[Tuple[str, str]]]:
"""
Find a path of converters to go from one version to another.
Args:
from_version (str): The source version
to_version (str): The target version
Returns:
list: A list of (from, to) version pairs representing the conversion path,
or None if no path is found
"""
if from_version == to_version:
return []
if (from_version, to_version) in _converters:
return [(from_version, to_version)]
queue = [(from_version, [])]
visited = {from_version}
while queue:
current, path = queue.pop(0)
for (src, dst), _ in _converters.items():
if src == current and dst not in visited:
new_path = path + [(src, dst)]
if dst == to_version:
return new_path
visited.add(dst)
queue.append((dst, new_path))
return None
def convert_backup(backup_dir: str, target_version: Optional[str] = None) -> str:
"""
Convert a backup to the target version.
Args:
backup_dir (str): Path to the backup directory
target_version (str, optional): The target version. If None, converts to the latest version.
Returns:
str: The new version of the backup
Raises:
ValueError: If no conversion path is found
"""
if target_version is None:
target_version = CURRENT_VERSION
current_version = get_backup_version(backup_dir)
if current_version == target_version:
return current_version
path = find_conversion_path(current_version, target_version)
if not path:
raise ValueError(f"No conversion path found from version {current_version} to {target_version}")
for from_ver, to_ver in path:
converter = _converters.get((from_ver, to_ver))
if converter:
logger.info(f"Converting backup from version {from_ver} to {to_ver}")
new_version = converter(backup_dir)
set_backup_version(backup_dir, new_version)
else:
raise ValueError(f"Converter not found for {from_ver} to {to_ver}")
return target_version
import pkgutil
import importlib
for _, name, is_pkg in pkgutil.iter_modules([os.path.dirname(__file__)]):
if not is_pkg and name != "__init__":
importlib.import_module(f"misc.backup_converters.{name}")

176
misc/backup_converters/v0_to_v1.py Normal file
View File

@@ -0,0 +1,176 @@
"""
Converter from version 0.0 to version 1.0
=========================================
This module converts backups from version 0.0 (old format without version info)
to version 1.0 (current format with version info).
The old version backup has a simple directory structure with .md files for markdown content,
.mdmeta files for markdown metadata, and .meta files for path metadata.
The current version backup has a more complex structure with tree/ and template/ directories,
.json files for markdown content, .json.meta files for path metadata, and a webhook.json file.
For all old version markdowns, they are considered to be using the default template with no other settings,
and their content is converted from plain text to {"markdown": content_in_md}.
For all old version paths, they are considered to have no settings.
"""
import os
import json
import shutil
from datetime import datetime
import logging
import tempfile
from misc.backup_converters import register_converter
logger = logging.getLogger(__name__)
def convert_v0_to_v1(backup_dir: str) -> str:
"""
Convert a backup from version 0.0 to version 1.0.
Args:
backup_dir (str): Path to the backup directory
Returns:
str: The new version ("1.0")
"""
logger.info(f"Converting backup from version 0.0 to 1.0: {backup_dir}")
temp_dir = tempfile.mkdtemp()
new_backup_dir = os.path.join(temp_dir, "new_backup")
os.makedirs(new_backup_dir)
try:
tree_dir = os.path.join(new_backup_dir, 'tree')
template_dir = os.path.join(new_backup_dir, 'template')
os.makedirs(tree_dir)
os.makedirs(template_dir)
with open(os.path.join(new_backup_dir, 'webhook.json'), 'w') as f:
json.dump([], f)
with open(os.path.join(new_backup_dir, 'version.json'), 'w') as f:
json.dump({"version": "1.0"}, f)
root_dir = os.path.join(backup_dir, 'Root')
if os.path.exists(root_dir) and os.path.isdir(root_dir):
source_dir = root_dir
else:
source_dir = backup_dir
convert_directory(source_dir, tree_dir)
for item in os.listdir(new_backup_dir):
src = os.path.join(new_backup_dir, item)
dst = os.path.join(backup_dir, item)
if os.path.isdir(src):
if os.path.exists(dst):
shutil.rmtree(dst)
shutil.copytree(src, dst)
else:
shutil.copy2(src, dst)
shutil.rmtree(temp_dir)
return "1.0"
except Exception as e:
logger.error(f"Error converting backup: {e}")
shutil.rmtree(temp_dir)
raise
def convert_directory(old_dir: str, new_dir: str):
"""
Recursively convert a directory from the old backup format to the new format.
This function processes a directory from the old backup format and converts it to the new format.
It creates a .json.meta file for the directory with metadata from the .meta file (if it exists),
processes all markdown files (.md) in the directory, converting them to .json files with the new format,
and recursively processes all subdirectories.
For markdown files, the function:
1. Reads the content from the .md file
2. Converts the content to the new format {"markdown": content_in_md}
3. Reads metadata from the .mdmeta file (if it exists)
4. Creates a new .json file with the content and metadata
For directories, the function:
1. Reads metadata from the .meta file (if it exists)
2. Creates a new .json.meta file with the metadata
3. Recursively processes all subdirectories
Args:
old_dir (str): Path to the old directory
new_dir (str): Path to the new directory where the converted files will be stored
"""
order = ''
meta_file_path = os.path.join(old_dir, '.meta')
if os.path.exists(meta_file_path):
with open(meta_file_path, 'r') as f:
for line in f:
try:
if line.startswith('order:'):
order = line.strip().split(': ', 1)[1]
except (ValueError, IndexError):
continue
path_meta = {
'name': os.path.basename(old_dir),
'order': order,
'backup_id': 0
}
with open(os.path.join(new_dir, '.json.meta'), 'w') as f:
json.dump(path_meta, f, default=str, indent=2)
for file_name in os.listdir(old_dir):
old_file_path = os.path.join(old_dir, file_name)
if file_name.endswith('.md'):
md_title = file_name[:-3]
with open(old_file_path, 'r', encoding='utf-8') as f:
content = f.read()
new_content = json.dumps({'markdown': content})
created_at = datetime.now()
order = ''
shortcut = ''
mdmeta_path = old_file_path + 'meta'
if os.path.exists(mdmeta_path):
with open(mdmeta_path, 'r') as f:
for line in f:
try:
key, value = line.strip().split(': ', 1)
if key == 'created_at':
created_at = value
elif key == 'order':
order = value
elif key == 'shortcut':
shortcut = value
except ValueError:
continue
md_data = {
'title': md_title,
'content': new_content,
'created_at': created_at,
'order': order,
'shortcut': shortcut,
'backup_id': 0
}
with open(os.path.join(new_dir, f'{md_title}.json'), 'w') as f:
json.dump(md_data, f, default=str, indent=2)
elif os.path.isdir(old_file_path) and not file_name.startswith('.'):
new_subdir = os.path.join(new_dir, file_name)
os.makedirs(new_subdir, exist_ok=True)
convert_directory(old_file_path, new_subdir)
register_converter("0.0", "1.0", convert_v0_to_v1)