# "*": "https://raw.githubusercontent.com/wefindx/schema/master/intent/oo-item.yaml" # "base:title": "0oo - Data Liquidity and Systems Interoperability" "og:title": "Data Liquidity and Systems Interoperability" "og:description": "Data integration and analytics is a bottleneck for solving our greatest challenges from doing science and creating general artificial intelligence, to everything in between. The demand for integrated data is indicated by the number of startups that focus on nothing more than collecting lists of well-aligned data-sets of interest and monetizing specialized queries. Well-aligned quality datasets is the gold-mine for endeavors involving inherently heterogeneous data, such as for drug discovery, complex designs, sociological research, and so on. Presence of multitude of data formats and standards makes any simple question, such as "get me a list of all world's dogs" - …" "og:image": "https://avatars0.githubusercontent.com/u/28134655" "og:url": "/intent/1311/" "base:css": "/static/css/bootstrap.min.9c25540d6272.css" "base:extra-css": "/static/css/base.57997aeac1df.css" "base:favicon": "/static/favicon.acaa334f0136.ico" "base:body_class": "" "layout:logo": "/static/0oo.8d2a8bbef612.svg" "layout:index": "/" "layout:menu": "/menu/" "layout:categories": "/intents/" "layout:ideas": "/methods/" "layout:projects": "/projects/" "layout:users": "/users/" "layout:about": "/about/" "layout:help": "/help/" "layout:bug_report": "https://github.com/wefindx/0oo" "layout:login": "/accounts/login/" "layout:light-off": "/darken/?darken=true" "layout:set-multilingual": "/mulang/?mulang=true" "layout:lang": "Language" "layout:set-language-post-action": "/i18n/setlang/" "layout:csrf-token": "66Bk3agGaXadhK9fhzJtHHW68SFHtnHzw0si4Wcyuy79j1jgGePcpOS1Gy1ZYhoc" "layout:input-next": "/intent/1311/" "layout:languages": [{"code": "ja", "is-active": "false", "name": "日本語"}, {"code": "lt", "is-active": "false", "name": "Lietuviškai"}, {"code": "zh-hans", "is-active": "false", "name": "简体中文"}, {"code": "en", "is-active": "true", "name": "English"}, {"code": "ru", "is-active": "false", "name": "Русский"}, {"code": "oo", "is-active": "false", "name": "O;o,"}] # "item:parent:intents": [{"title": "Understanding Web Data", "url": "/intent/778/"}, {"title": "Meta standard for data reusability", "url": "/intent/862/"}, {"title": "Big Data and Data Science", "url": "/intent/1203/"}] "item:title": ".:en:Data Liquidity and Systems Interoperability.:cn:数据流动性和系统互操作性" "item:votes": +1 "item:add-vote": "#addnote" "item:intent": "/intent/1311/?l=en" "item:base-administration": false "item:body": | .:en Data integration and analytics is a bottleneck for solving our greatest challenges from doing science and creating general artificial intelligence, to everything in between. The demand for integrated data is indicated by the number of startups that focus on nothing more than collecting lists of well-aligned data-sets of interest and monetizing specialized queries. Well-aligned quality datasets is the gold-mine for endeavors involving inherently heterogeneous data, such as for drug discovery, complex designs, sociological research, and so on. Presence of multitude of data formats and standards makes any simple question, such as "get me a list of all world's dogs" - an insurmountable quest for yet another startup focusing on that specific domain. The existing solutions, such as linked ontology-aware data formats are insufficiently flexible and rich to be convenient for defining records with multi-vocabulary fields from arbitrary ad-hoc vocabularies, and lack support for definitions of value types, callable object interfaces and modification permissions, enabling objects to retain properties even after decoupling from the data management systems that originate them. Current widely known solutions (such as [Linked Data](https://www.w3.org/standards/semanticweb/data)), are not entirely well suited for the problem, as they require large amounts of data to be serialized in the same format, that never is the case in the ever diversifying world, and there is no standard way to embed schemas, permissions and other context data to data items, necessary to make them reusable in queries. Combining the RDF-based SPARQL (for alignment) with OAUTH2 (for permissioning) and some and a standard to securely encrypt data about query origin context (such as query origin identity keys, cookies, IP addresses, and definitions of schema versions of resources, where data came from) it may be possible to approach the desired data properties of retaining the ability to reuse data items as objects in the context of arbitrary programming languages, without the need to write custom integrations. However, this seem to have not been done, and there may be better solutions to address the problem. For example, due to the diversity and complexity of systems on the web (protocols and formats), there may be other (better?) ways to approach the problem, based on plug-and-play philosophy for devices using drivers, allowing to abstract away web resource APIs, and have fully-featured polymorphic interactive data as a shared feature of all programming languages, treating websites and web systems (including decentralized ones) as operating system devices directly available as variables to programming languages. Regardless of the choice or way of implementation, the data liquidity and systems interoperability seem to remain an important unsolved problem and bottleneck for faster progress in large number of domains of digital activity. .:cn 数据集成和分析是解决我们最大挑战的瓶颈,从做科学和创造通用人工智能到介于两者之间的一切。对集成数据的需求,可以从一些创业公司的数量上看出来,这些公司只专注于收集感兴趣的整齐划一的数据集清单,并将专项查询货币化。对于涉及内在异质数据的工作来说,如药物发现、复杂设计、社会学研究等,高度一致的高质量数据集是金矿。众多数据格式和标准的存在,使任何如"列出世界上所有狗的名单"之类简单的问题,对于专注于其它特定领域的创业公司来说,是一个难以逾越的追求。现有的解决方案,如链接的本体感知数据格式不够灵活和丰富,无法方便地从任意特设词汇表中定义具有多词汇字段的记录,而且缺乏对值类型的定义、可调用对象接口和修改权限的支持,使得对象无法在与产生它们的数据管理系统解耦后仍能保留属性。 当前广为人知的解决方案(如“[链接数据](https://www.w3.org/standards/semanticweb/data)”),并不完全适合这个问题,因为它们需要将大量数据以相同格式序列化,而在日益多样化的世界中从来没有这种情况,而且没有标准的方法将模式、权限和其他上下文数据嵌入到数据项中,这是使它们在查询中可以重用的必要条件。 将基于RDF的SPARQL(用于对齐)与OAUTH2(用于许可)和一些与标准结合以安全地加密关于查询源上下文的数据(如查询源身份密钥、Cookie、IP地址和资源的架构版本的定义,数据来自何处),它可能会接近所需的数据属性,即保留在任意编程语言的上下文中作为对象重用数据项的能力,而不需要编写自定义集成。然而这似乎尚未完成,可能有更好的方案来解决该问题。 例如,由于网络中系统的多样性和复杂性(协议和格式),可能还有其他(更好的?)方法来解决这个问题,基于使用驱动程序的设备的即插即用原理,允许抽象化Web资源API,并将功能齐全的多态交互数据作为所有编程语言的共享功能,将网站和Web系统(包括去中心化系统)作为操作系统设备直接作为变量提供给编程语言。 无论选择何种实现方式,数据流动性和系统互操作性似乎仍然是重要的未解决问题,也是在众多数字活动领域中快速发展的瓶颈。 "item:permalink": "/intent/1311/?l=en" "item:source-date": "" "item:owner": "Mindey" "item:ownerlink": "/user/147/Mindey" "item:created": "2020-09-13T00:20:01.169000" "item:intent:child:add": "/admin/hlog/intent/add/?parent=1311" # "item:method:items": - "id": "i-863" "title": "Metaformat" "summary": "Create a polycontext metasymbol, and overcome the fact that standardization does not generalize." "url": "/method/863/" "owner": "Mindey" "permalink": "/intent/1311/?l=en#i-863" "ownerlink": "/user/147/Mindey" "created": "2019-09-11T15:01:44.134000" - "id": "i-982" "title": "Create a list of join queries that would be useful cross institutions" "summary": "In designing this data liquidity system, we should have some idea of the kinds of questions we want to answer" "url": "/method/982/" "owner": "chronological" "permalink": "/intent/1311/?l=en#i-982" "ownerlink": "/user/198/chronological" "created": "2020-09-13T16:39:13.663000" - "id": "i-986" "title": "Automated API traversal" "summary": "Armed with a thesaurus and an almanac of system functionality we can write robots that program themselves" "url": "/method/986/" "owner": "chronological" "permalink": "/intent/1311/?l=en#i-986" "ownerlink": "/user/198/chronological" "created": "2020-09-26T22:40:04.921000" "item:method:add": "/admin/hlog/method/add/?parent=1311" "item:comment:add": "/intents/addnote?parent=1311" "item:comment:add:csrf_token": "66Bk3agGaXadhK9fhzJtHHW68SFHtnHzw0si4Wcyuy79j1jgGePcpOS1Gy1ZYhoc" "item:comment:form": |
  • Mark if the comment raises new questions.
  • Mark if the comment contributes potential solutions.
  • Mark if the comment contributes facts for reasoning.
  • Please, log in. # "item:comment:items": - "id": "r-133" "text": | I'm upvoting this as it's something I want to happen too. I see two ways of it happening: * People cooperate and create integrations. * People don't cooperate and don't create integrations. In which case we have to take matters into our own hands. I believe keylogging at the operating system or browser level is the only way to keep ownership of our own data. "owner": "chronological" "ownerlink": "/user/198/chronological" "permalink": "/intent/1311/?l=en#r-133" "created": "2020-10-01T16:52:33.041000" "vote": "[+]" "base:js": "/static/js/base.c7357c06cc89.js"