#
"*": "https://raw.githubusercontent.com/wefindx/schema/master/project/oo-item.yaml"
#
"base:title": "0oo - Postgres synchronization solution"
"og:title": "Postgres synchronization solution"
"og:description": "This is a challenging computer problem, data synchronization but I have an idea I plan to use a inefficient rolling hash and sorting to solve the problem of synchronization. This is similar to rsync"
"og:image": "https://avatars0.githubusercontent.com/u/28134655"
"og:url": "/intent/52001/"
"base:css": "/static/css/bootstrap.min.9c25540d6272.css"
"base:extra-css": "/static/css/base.57997aeac1df.css"
"base:favicon": "/static/favicon.acaa334f0136.ico"
"base:body_class": ""
"layout:logo": "/static/0oo.8d2a8bbef612.svg"
"layout:index": "/"
"layout:menu": "/menu/"
"layout:categories": "/intents/"
"layout:ideas": "/methods/"
"layout:projects": "/projects/"
"layout:users": "/users/"
"layout:about": "/about/"
"layout:help": "/help/"
"layout:bug_report": "https://github.com/wefindx/0oo"
"layout:login": "/accounts/login/"
"layout:light-off": "/darken/?darken=true"
"layout:set-multilingual": "/mulang/?mulang=true"
"layout:lang": "Language"
"layout:set-language-post-action": "/i18n/setlang/"
"layout:csrf-token": "hh3DTbngWO8OEwyjz79B5Kr5ct5mEF6IEOFJWCgg2TxnPFK9nPKNc1PUsR49RHpf"
"layout:input-next": "/project/52001/"
"layout:languages": [{"code": "ja", "is-active": "false", "name": "日本語"}, {"code": "lt", "is-active": "false", "name": "Lietuviškai"}, {"code": "zh-hans", "is-active": "false", "name": "简体中文"}, {"code": "en", "is-active": "true", "name": "English"}, {"code": "ru", "is-active": "false", "name": "Русский"}, {"code": "oo", "is-active": "false", "name": "O;o,"}]
#
"item:title": "Postgres synchronization solution"
"item:score": -8.0
"item:mission": "I want to take one set of data and synchronize it with another set where there could be changed to columns, new rows and deleted rows"
"item:permalink": "/project/52001/?l=en"
"item:base-administration": false
"item:body": |
This is a challenging computer problem, data synchronization but I have an idea
I plan to use a inefficient rolling hash and sorting to solve the problem of synchronization. This is similar to rsync
"item:destinations": |
"item:source-date": ""
"item:permalink": "/method/52001/?l=en"
"item:owner": "chronological"
"item:created": "2022-08-16T13:27:07.939085"
"item:ownerlink": "/user/198/chronological"
#
"item:result:items":
- "id": "t-146001"
"url": "https://github.com/samsquire/postgres-sync"
"text": |
I created a vagrant setup with 3 postgres installations and created a cronjob to run the sync script
"cost": -5.0
"currency": "HUR"
"assets": |
My vagrant setup uses persistent disks and uses ansible to deploy the cronjob and sync script. It is configured by YAML file. I've also installed psycopg2 and I found documentation on how to retrieve the tables in a database in Postgres. It's just a matter of writing the sync algorithm now.
My problem is detecting which side is the winning copy.
When one side changes the data there shall be a different hash and the changed rows are detected. This part I understand.
The problem is detecting which side is the latest change and which side should win. I might need to introduce a version column.
If I had a last updated timestamp field I could use that. Or a version column but I am expressly trying to avoid introducing new columns to the schema. It means it's a lot harder.
"owner": "chronological"
"ownerlink": "/user/198/chronological"
"permalink": "/project/52001/?l=en#t-146001"
"created": "2022-08-16T13:32:29.650418"
"happened": "2022-08-16T13:28:10"
"models":
- "id": "t-151001"
"url": "https://github.com/samsquire/postgres-sync"
"text": |
I implemented column hashing
"cost": -3.0
"currency": "HUR"
"assets": |
I implemented a hash that takes previous data and current column and the previous hash to produce a hash of the entire database.
This allows us to synchronize with the minimum of data transmissions when I write the synchronizer part which shall rehash all its own data, then retrieve the hash of a binary search of the sorted data.
"owner": "chronological"
"ownerlink": "/user/198/chronological"
"permalink": "/project/52001/?l=en#t-151001"
"created": "2022-09-09T20:23:41.331046"
"happened": "2022-09-09T20:21:55"
"models":
"item:result:add": "/admin/hlog/task/add/?parent=52001"
"item:transfer:items":
"item:transfer:add": "/admin/hlog/operation/add/?parent=52001"
"item:comment:add": "/projects/addnote?parent=52001"
"item:comment:add:csrf_token": "hh3DTbngWO8OEwyjz79B5Kr5ct5mEF6IEOFJWCgg2TxnPFK9nPKNc1PUsR49RHpf"
"item:comment:form": |
Mark if the comment raises new questions.
Mark if the comment contributes potential solutions.
Mark if the comment contributes facts for reasoning.
Mark if the comment contributes tasks for doing.
Please, log in.
#
"comments":
"base:js": "/static/js/base.c7357c06cc89.js"