From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 4A121B4B for ; Fri, 7 Apr 2017 21:44:45 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.7.6 Received: from out1-smtp.messagingengine.com (out1-smtp.messagingengine.com [66.111.4.25]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id B1556107 for ; Fri, 7 Apr 2017 21:44:44 +0000 (UTC) Received: from compute2.internal (compute2.nyi.internal [10.202.2.42]) by mailout.nyi.internal (Postfix) with ESMTP id E301820AA4 for ; Fri, 7 Apr 2017 17:44:43 -0400 (EDT) Received: from web3 ([10.202.2.213]) by compute2.internal (MEProxy); Fri, 07 Apr 2017 17:44:43 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; bh=tlJtWc 6pCZsH1BRj/IOn8rB9e5B90to/dyjP1imEObY=; b=lvtuHHsrHDQ7zQq+/LFPfc Ydmqrw9qYdmEQ04+NfnK5vCXPBgYbTfejvIBhNS9gHio+R6gHTzoKYnx7bh19ocF isyzdKPRLf5ox0K9dKPuuWtfev+8Lo/klhpX/3ubx02/genNhlMf++Mbp4TcopsT nftvRS7Nu9NuJN4GfzHLeekAOsMNXRdK0jfdmVgKpBDSgCqkn8CmxSEd1wFhbQ4i nRp0B38EquP4xSkyF75wV6JpQGCRJzQMyY/2oG7FirzhLpobaQtQsr+0AaHqZbWa YJ1RIZp6b7tum1nGPSlzO1yNIlvljQwv7MlvyYd5CB7gv+c/upGH0Gyk76hDBYPg == X-ME-Sender: Received: by mailuser.nyi.internal (Postfix, from userid 99) id C3F519EC4C; Fri, 7 Apr 2017 17:44:43 -0400 (EDT) Message-Id: <1491601483.1253274.937963928.1B1D9B41@webmail.messagingengine.com> From: Tomas To: bitcoin-dev@lists.linuxfoundation.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="utf-8" X-Mailer: MessagingEngine.com Webmail Interface - ajax-7c174d5d References: <1491516747.3791700.936828232.69F82904@webmail.messagingengine.com> Date: Fri, 07 Apr 2017 23:44:43 +0200 In-Reply-To: X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Fri, 07 Apr 2017 22:11:15 +0000 Subject: Re: [bitcoin-dev] Using a storage engine without UTXO-index X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 07 Apr 2017 21:44:45 -0000 Hi Eric, On Fri, Apr 7, 2017, at 21:55, Eric Voskuil via bitcoin-dev wrote: > Optimization for lower memory platforms then becomes a process of > reducing the need for paging. This is the purpose of a cache. The seam > between disk and memory can be filled quite nicely by a small amount > of cache. On high RAM systems any cache is actually a de-optimization > but on low RAM systems it can prevent excessive paging. This is > directly analogous to a CPU cache. I am not entirely sure I agree with that, or understand it correctly. If -for example - the data of some application is a set of records which can be sorted from least frequently used to most frequently used then doing just that sort will beat any application-layer cache. Regardless of size of data and size of RAM, you simply allow the OS to use disk caching or memory map caching to work its magic . In fact, I would argue that an application-layer cache *only* makes sense if the data model shows a *hard* distinction between often and not often used data. If usage-frequency is a continuous line, caching is best left to the OS by focussing on proper spatial and temporal locality of reference of your data, because the OS has much more information to make the right decision.