Storage & Migration¶

Vector Database Migration¶

Migrate from Weaviate to another database¶

Update configuration

Source code deployment (.env):

VECTOR_STORE=qdrant

Docker Compose (docker-compose.yaml):

VECTOR_STORE: qdrant

Run migration

# Source code
flask vdb-migrate

# Docker
docker exec -it docker-api-1 flask vdb-migrate

Tested databases: Qdrant, Milvus, AnalyticDB

Storage Migration¶

Move from local to cloud storage¶

Migrate files from local storage to cloud providers (e.g., Alibaba Cloud OSS):

Configure cloud storage

.env or docker-compose.yaml:

STORAGE_TYPE=aliyun-oss
# Add OSS credentials

Migrate data

Source code:

flask upload-private-key-file-to-cloud-storage
flask upload-local-files-to-cloud-storage

Docker:

docker exec -it docker-api-1 flask upload-private-key-file-to-cloud-storage
docker exec -it docker-api-1 flask upload-local-files-to-cloud-storage

Data Cleanup¶

Delete old logs¶

Get tenant ID

docker exec -it docker-api-1 bash -c "echo 'from models import Tenant; db.session.query(Tenant.id, Tenant.name).all(); quit()' | flask shell"

Delete logs older than X days

docker exec -it docker-api-1 flask clear-free-plan-tenant-expired-logs \
  --days 30 \
  --batch 100 \
  --tenant_ids 618b5d66-a1f5-4b6b-8d12-f171182a1cb2

Remove exported logs (optional)

docker exec -it docker-api-1 bash -c 'rm -rf ${OPENDAL_FS_ROOT}/free_plan_tenant_expired_logs'

Remove orphaned files¶

Warning: Back up database and storage before running. Run during maintenance window.

Clean database records

docker exec -it docker-api-1 flask clear-orphaned-file-records
# Use -f flag to skip confirmation

Delete orphaned files from storage

docker exec -it docker-api-1 flask remove-orphaned-files-on-storage
# Use -f flag to skip confirmation

Note: Only works with OpenDAL storage (STORAGE_TYPE=opendal).

Backup & Recovery¶

Create backup before upgrade¶

cp -r dify "dify.bak.$(date +%Y%m%d%H%M%S)"

What to backup¶

For Docker Compose deployments: - Entire flexai/docker/volumes directory

For source deployments: - Database - Storage configuration - Vector database data - Environment files

Database maintenance¶

After deleting logs, reclaim storage:

PostgreSQL:

VACUUM FULL;

Upgrade Process¶

Version upgrade¶

Image deployment:

docker compose pull
docker compose up -d

Source code:

git pull
cd api
flask db upgrade

Database schema migration¶

Always required for source code updates:

cd api
flask db upgrade