From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.osuosl.org (smtp1.osuosl.org [140.211.166.138]) by lists.linuxfoundation.org (Postfix) with ESMTP id 03EC5C002D for ; Fri, 26 Aug 2022 06:06:43 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp1.osuosl.org (Postfix) with ESMTP id BF17F83422 for ; Fri, 26 Aug 2022 06:06:42 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org BF17F83422 Authentication-Results: smtp1.osuosl.org; dkim=pass (2048-bit key, unprotected) header.d=notatether.com header.i=@notatether.com header.a=rsa-sha256 header.s=protonmail header.b=G/TzwLU9 X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -1.6 X-Spam-Level: X-Spam-Status: No, score=-1.6 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, PDS_BTC_ID=0.499, PDS_BTC_MSGID=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no Received: from smtp1.osuosl.org ([127.0.0.1]) by localhost (smtp1.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FmaG8_eCB_XD for ; Fri, 26 Aug 2022 06:06:41 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp1.osuosl.org E02368336F Received: from mail-40136.proton.ch (mail-40136.proton.ch [185.70.40.136]) by smtp1.osuosl.org (Postfix) with ESMTPS id E02368336F for ; Fri, 26 Aug 2022 06:06:39 +0000 (UTC) Date: Fri, 26 Aug 2022 06:06:30 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=notatether.com; s=protonmail; t=1661493996; x=1661753196; bh=zKYwQtUa8ChGC3ZRXQTjxdp8tSwzqcIqTJ4M1I0bi+c=; h=Date:To:From:Cc:Reply-To:Subject:Message-ID:In-Reply-To: References:Feedback-ID:From:To:Cc:Date:Subject:Reply-To: Feedback-ID:Message-ID; b=G/TzwLU9hoyTZJbI16w47ML/q4QLDm+6u+0mLkKVWZt8tIsV7Y+1n2CGyGQf7T8Yi XCdcJL/EORQK0EVnj401OKKgXT7WobjZJS8CGU9DYiYi0eYWS8xnceHjoHsqk+5z+3 LSNmIjR9uisCnRIP1OJ+XeS27a1fLwLh9bZVFKW7sGUq4NSkodbFLpxGeTEKkAkHTW aQSZjJ/3JvJpKBWIiT4oasbD3mCwLlhxn+9UtOgDIMk9w/mBNlSpe9KU0dL1Uqk02O UJOsUZKjlncrQZ1e30cCGxYX2M/J1yveeDAFhbA9EvXkIlWsjg51WS3Rpx7H1F4Vg8 z9aVrbXlU6umQ== To: rhavar@protonmail.com From: Ali Sherief Reply-To: Ali Sherief Message-ID: <20220826060620.mgpbmnpw7cwea3az@artanis> In-Reply-To: References: Feedback-ID: 34210769:user:proton MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Mailman-Approved-At: Fri, 26 Aug 2022 07:46:23 +0000 Cc: bitcoin-dev@lists.linuxfoundation.org Subject: Re: [bitcoin-dev] BIP Proposal: Wallet Labels Export Format X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 26 Aug 2022 06:06:43 -0000 I think these problems can be mitigated if the CSV format is strictly defin= ed, such as how I specified it in my previous message. In particular, the parser has to recognize only one specific header line th= at has a version number somewhere, or abort - and I still insist on quoting= the labels with double-quote and introducing a 3rd column with specific st= ring or numeric types and then replacing all the special characters in the = input/output with ":". Strictly defining CSV version and consequentially, the fields, and then spe= cifying on what kind of data the import is supposed to fail at will limit t= he complexity of importers to N different switch cases - where N is the num= ber of circulating versions of the format (for now 1). - Ali On Thu, Thu, 25 Aug 2022 13:48:36 +0000, rhavar@protonmail.com wrote: > > Not only is JSON limited to editing only through specific software or t= ext editors, but (in the latter case) it is fragile enough that a single mi= ssing character can cause an entire file to fail parsing. CSV is more forgi= ving in this regard. > > I think quite simply: A forgiving format is not appropriate for a standar= d. > > It'd be hard to understate how much extra and pointless effort it creates= for everyone, and every implementation ends up creating its own defacto st= andard for what it produces and accepts. Even doing something as simple as = adding an extra column will not be possible in the future because it'll bre= ak comparability with previous parsers. > > I've literally worked on projects where the csv parser has evolved into s= can-ahead to use heuristics to understand "rules" of a csv file, and then d= o line-by-line heuristics to override those rules in pathological cases. Ma= kes a bit of sense when you're trying to achieve 30 years of backwards comp= atibility. Doesn't make sense for much else.. > > If your application users really like csv, then introduce an application-= specific import-from-csv and export-to-csv with your own rules. > -Ryan > > ------- Original Message ------- > On Thursday, August 25th, 2022 at 1:59 AM, Craig Raw = wrote: > > > Thanks for your thoughts Ryan. > > > > Without reference to the quality feedback on this proposal, I was aware= when submitting it for review that it provides an excellent opportunity fo= r bike shedding. As developers, we have all experienced frustration with da= ta formats. One thing that I did not perhaps make clear enough is that this= format is not solely intended for developers, but general users who are pr= obably not well represented on this list. > > > > While doing research for this proposal I spoke to several professional = users of Sparrow Wallet (who are not developers). They all expressed a desi= re for the format to integrate with their business processes, which are dri= ven by business tools such as Excel. Labelling provides an important functi= on in UTXO and address management in these scenarios, and needs to be acces= sible and manageable outside of wallet software. > > > > If this is to be achieved, it immediately rules out JSON as a data form= at. Not only is JSON limited to editing only through specific software or t= ext editors, but (in the latter case) it is fragile enough that a single mi= ssing character can cause an entire file to fail parsing. CSV is more forgi= ving in this regard. With respect to your comments on escaping, my expectat= ion would be that developers will be using a mature CSV library rather than= handling character escaping themselves. I would rather propose a format th= at is generally usable, even if occasionally a label is escaped incorrectly= . > > > > Finally, I'll note that CSV files are already common and uncontroversia= l in Bitcoin wallet software. Bitcoin Core, Electrum, Sparrow (and no doubt= many others) already export addresses and/or transactions with their label= s as CSV files. This proposal simply attempts to create a standard for impo= rting and exporting all the labels in a wallet. > > > > Craig > > > > On Wed, Aug 24, 2022 at 9:01 PM wrote: > > > >> I'd strongly suggest not using CSV. Especially for a standard. I've wo= rked with it as an interchange format many a times, and it's always been a = clusterfuck. > >> > >> Right off the bat, you have stuff like "The fields may be quoted, but = this is unnecessary as the first comma in the line will always be the delim= iter" which invariably leads to some implementations doing it, some impleme= ntations not doing it, and others that are intolerant of the other way. > >> > >> And you have also made the classic mistake of not strictly defining es= cape rules. So everyone will pick their own (e.g. some will \, escape comma= s, others will not cause it's quoted and escape quotes, and others will ass= ume no escaping is required since its the last column in a csv). > >> > >> Over time it morphs into its own mini-monster that introduces so much = pain. > >> > >> On a similar note, allowing alternatives (like: txid>index vs txid:ind= ex) provides no benefit, but creates additional work for implementations (w= ho quite likely only test formats they produce) and future incompatibilitie= s. > >> > >> I know everyone loves to hate on it, but really (line-separated?) json= is the way to go. > >> > >> { "tx": "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b= 37b?", "label": "wow, such label" } > >> { "tx: "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b3= 7b", "txout": 4, "label": "omg this is so easy to parse" } > >> { "tx: "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b3= 7b", "txin": 0, "label": "wow this is going to be extensible as well" } > >> > >> -Ryan > >> > >> ------- Original Message ------- > >> On Wednesday, August 24th, 2022 at 2:18 AM, Craig Raw via bitcoin-dev = wrote: > >> > >>> Hi all, > >>> > >>> I would like to propose a BIP that specifies a format for the export = and import of labels from a wallet. While transferring access to funds acro= ss wallet applications has been made simple through standards such as BIP39= , wallet labels remain siloed and difficult to extract despite their value,= particularly in a privacy context. > >>> > >>> The proposed format is a simple two column CSV file, with the referen= ce to a transaction, address, input or output in the first column, and the = label in the second column. CSV was chosen for its wide accessibility, espe= cially to users without specific technical expertise. Similarly, the CSV fi= le may be compressed using the ZIP format, and optionally encrypted using A= ES. > >>> > >>> The full text of the BIP can be found at https://github.com/craigraw/= bips/blob/master/bip-wallet-labels.mediawiki and also copied below. > >>> > >>> Feedback is appreciated. > >>> > >>> Thanks, > >>> Craig Raw > >>> > >>> --- > >>> > >>> 
> >>> BIP: wallet-labels
> >>> Layer: Applications
> >>> Title: Wallet Labels Export Format
> >>> Author: Craig Raw 
> >>> Comments-Summary: No comments yet.
> >>> Comments-URI: https://github.com/bitcoin/bips/wiki/Comments:BIP-walle=
t-labels
> >>> Status: Draft
> >>> Type: Informational
> >>> Created: 2022-08-23
> >>> License: BSD-2-Clause
> >>>
> >>> > >>> =3D=3DAbstract=3D=3D > >>> > >>> This document specifies a format for the export of labels that may be= attached to the transactions, addresses, input and outputs in a wallet. > >>> > >>> =3D=3DCopyright=3D=3D > >>> > >>> This BIP is licensed under the BSD 2-clause license. > >>> > >>> =3D=3DMotivation=3D=3D > >>> > >>> The export and import of funds across different Bitcoin wallet applic= ations is well defined through standards such as BIP39, BIP32, BIP44 etc. > >>> These standards are well supported and allow users to move easily bet= ween different wallets. > >>> There is, however, no defined standard to transfer any labels the use= r may have applied to the transactions, addresses, inputs or outputs in the= ir wallet. > >>> The UTXO model that Bitcoin uses makes these labels particularly valu= able as they may indicate the source of funds, whether received externally = or as a result of change from a prior transaction. > >>> In both cases, care must be taken when spending to avoid undesirable = leaks of private information. > >>> Labels provide valuable guidance in this regard, and have even become= mandatory when spending in several Bitcoin wallets. > >>> Allowing users to export their labels in a standardized way ensures t= hat they do not experience lock-in to a particular wallet application. > >>> In addition, by using common formats, this BIP seeks to make manual o= r bulk management of labels accessible to users without specific technical = expertise. > >>> > >>> =3D=3DSpecification=3D=3D > >>> > >>> In order to make the import and export of labels as widely accessible= as possible, this BIP uses the comma separated values (CSV) format, which = is widely supported by consumer, business, and scientific applications. > >>> Although the technical specification of CSV in RFC4180 is not always = followed, the application of the format in this BIP is simple enough that c= ompatibility should not present a problem. > >>> Moreover, the simplicity and forgiving nature of CSV (over for exampl= e JSON) lends itself well to bulk label editing using spreadsheet and text = editing tools. > >>> > >>> A CSV export of labels from a wallet must be a UTF-8 encoded text fil= e, containing one record per line, with records containing two fields delim= ited by a comma. > >>> The fields may be quoted, but this is unnecessary, as the first comma= in the line will always be the delimiter. > >>> The first line in the file is a header, and should be ignored on impo= rt. > >>> Thereafter, each line represents a record that refers to a label appl= ied in the wallet. > >>> The order in which these records appear is not defined. > >>> > >>> The first field in the record contains a reference to the transaction= , address, input or output in the wallet. > >>> This is specified as one of the following: > >>> * Transaction ID (txid) > >>> * Address > >>> * Input (rendered as txid) > >>> * Output (rendered as txid>index or txid:index) > >>> > >>> The second field contains the label applied to the reference. > >>> Exporting applications may omit records with no labels or labels of z= ero length. > >>> Files exported should use the .csv file extension. > >>> > >>> In order to reduce file size while retaining wide accessibility, the = CSV file may be compressed using the ZIP file format, using the .zip file extension. > >>> This .zip file may optionally be encrypted using either AES-= 128 or AES-256 encryption, which is supported by numerous applications incl= uding Winzip and 7-zip. > >>> In order to ensure that weak encryption does not proliferate, importe= rs following this standard must refuse to import .zip files encryp= ted with the weaker Zip 2.0 standard. > >>> The textual representation of the wallet's extended public key (as de= fined by BIP32, with an xpub header) should be used as the passwor= d. > >>> > >>> =3D=3DImporting=3D=3D > >>> > >>> When importing, a naive algorithm may simply match against any refere= nce, but it is possible to disambiguate between transactions, addresses, in= puts and outputs. > >>> For example in the following pseudocode: > >>> 
> >>> if reference length < 64
> >>> Set address label
> >>> else if reference length =3D=3D 64
> >>> Set transaction label
> >>> else if reference contains '<'
> >>> Set input label
> >>> else
> >>> Set output label
> >>>
> >>> > >>> Importing applications may truncate labels if necessary. > >>> > >>> =3D=3DTest Vectors=3D=3D > >>> > >>> The following fragment represents a wallet label export: > >>> 
> >>> Reference,Label
> >>> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b?,Tra=
nsaction
> >>> 1A69TXnEM2ms9fMaY9UuiJ7415X7xZaUSg,Address
> >>> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b?<0,I=
nput
> >>> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b?>0,O=
utput
> >>> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b?:0,O=
utput (alternative)
> >>>
> >>> > >>> =3D=3DReference Implementation=3D=3D > >>> > >>> TBD