The Power of Continued Innovation: MongoDB Adds Transactions

Yesterday I did a fireside chat at our portfolio company Shippo and someone asked the question how a company can continue to be successful and grow over a long period of time. My answer was: continue to innovate. A company that wonderfully represents that spirit is MongoDB. Eliot Horowitz, the co-founder and CTO of MongoDB, just announced in a blog post that MongoDB 4.0 will support multi-document transactions.

You should go read the post as it is great, including the epic title “MongoDB Drops ACID” (which I will explain later), but here is the quote that really stood out to me (emphases in bold are mine):

The imminent arrival of transactions is the culmination of a multi-year engineering effort, beginning over 3 years ago with the integration of the WiredTiger storage engine. We’ve laid the groundwork in almost every part of the server – from the storage layer itself, Continue reading "The Power of Continued Innovation: MongoDB Adds Transactions"

Uncertainty Wednesday: Spurious Correlation (Part 3)

So the last two Uncertainty Wednesday posts have been about spurious correlation. Today, I want to give an example of easy it is to observe spurious correlation. To that end I wrote a little Python program which I will show below that rolls two independent dice. Each is rolled 10 times to give us two data series of 10 points each. This mimics the 10 data point series from last week’s example.

The program runs some number of these and each times calculates and outputs the coefficient of correlation. I then use Google Sheets to produce a histogram.

Here is the result for 1,000 runs

image

And here are 10,000 runs

image

What we see is the distribution of the sample correlation. As we add more runs we once again see a normal distribution emerge (isn’t that fascinating?).

Looking at the charts we see that the center of the distribution is Continue reading "Uncertainty Wednesday: Spurious Correlation (Part 3)"

World After Capital: Digital Technology (Zero Marginal Cost)

NOTE: Last week’s blog post was the current introduction to my book World After Capital. Today’s post covers the first half of the chapter on Digital Technology, which discusses how zero marginal cost is unlike anything found in the analog world.

Digital Technology

The invention of agriculture expanded the space of the possible by dramatically increasing the food density of land. This allowed humanity to have surplus food, which provided the basis for increased population density and hierarchical societies that developed standing armies, specialization of labor and writing [5].

The Enlightenment and subsequent Industrial Revolution further expanded the space of the possible by substituting human power for machine power and increasing our understanding of, and control over, chemical and physical transformations of matter. This allowed humanity to make extraordinary material progress on the basis of innovations in energy, manufacturing, transportation and communication [6].

Digital technologies provide the third expansion of Continue reading "World After Capital: Digital Technology (Zero Marginal Cost)"

Uncertainty Wednesday: Spurious Correlation (Cont’d)

Last Uncertainty Wednesday, I introduced the topic of spurious correlation. Since then I have discovered a site that gives some fantastic examples of (potentially) spurious correlations. Here is one:

image

The coefficient of correlation is 0.9926, i.e. almost 1 (which would be perfectly correlated).

Let’s remind ourselves what finding correlation in a sample of data means. It is simply a numerical measure that can be computed for any paired data. The formula produces a result that has nothing to with the labels on the data. This may seem like stating the obvious, but it is really important to keep in mind. The numerical result for correlation here is the same whether the labels read “Divorce rate in Maine” and “Per capita consumption of margarine” or if they were simply “Series 1″ and “Series 2.″ 

Why am I emphasizing this? Because whether or not we think sample correlation is indicative of real Continue reading "Uncertainty Wednesday: Spurious Correlation (Cont’d)"

World After Capital: Introduction

NOTE: This is the current state of the introduction for World After Capital. It provides an overview of the key ideas from the book. In case you missed it, last week’s blog post has the work in progress note and the preface.

Introduction

Humanity is unique, at least for now, in having developed knowledge. Knowledge in turn has enabled us to create increasingly powerful technology. The effect of technological advances is to broaden the “space of the possible.”

  • With the Internet we can give everyone free access to education, but we can also share hate speech globally
  • With artificial intelligence we can build self-driving cars, but we can also automate censorship and manipulation

A broader space of the possible contains both good and bad capabilities. There is nothing fundamentally new about this duality of technology.

Uncertainty Wednesday: Spurious Correlation (Intro)

In the last Uncertainty Wednesday post on Sample Variance, I wrote that “Inference from data without explanations is how people go deeply wrong about reality.” It occurred to me that the best way to illustrate this is by writing about spurious correlation. To do that I first have to introduce the concept of correlation though. It may seem surprising that I have gotten this far into the series without doing so, but we spent a fair bit of time on a related concept, namely independence.

If you don’t recall, you should go back and read the posts on independence. The opposite of independence of two (or more) random variables is dependence. Now this is where it gets confusing. Sometimes the word “correlation” is used as a synonym for “dependence.” But more commonly “correlation” refers to a measure of a specific type of dependence, namely linear dependence.

The Wikipedia entry on correlation

Continue reading "Uncertainty Wednesday: Spurious Correlation (Intro)"

World After Capital: Work In Progress & Preface

NOTE: One of my goals for 2018 is to bring my book World After Capital to the point where I feel it is good enough to publish in paper form. I have come to realize that if I just keep blogging about other topics at the same time I won’t get there. So going forward every Monday I am planning to post revised pages or whole sections of the book starting today with the sections titled “Work in Progress” and “Preface.”

Work in Progress

This book is a work in progress. What you are reading now is a draft with known problems and placeholders. It does, however, include all the major ideas and what remains is a process of gradual improvement.

The process of writing in this way is an example of what I call the “knowledge loop” in the book. The knowledge loop consists of learning, creating Continue reading "World After Capital: Work In Progress & Preface"

Mycroft: Open Consumer Voice Platform

Late in 2016 I wrote a blog post titled “Voice Platforms: Open Alternative is an Opportunity.” At the time I learned about a project called Mycroft. This project has just launched its second version, the Mark II, on Kickstarter. If you want voice in your home but are weary of Amazon and Google (and the other tech giants), this looks like an interesting alternative. Here is their intro video, which is super funny in a self deprecating way:

I just went ahead and backed the project and look forward to trying out the Mark II at home. I don’t know how many users such a platform will need to be viable but we won’t know unless we try!

Uncertainty Wednesday: Sample Variance (Cont’d)

Last Uncertainty Wednesday, I used meteorite impact data to make the point that sample variance may be much smaller than actual variance. Following the post, I was asked a great question on Twitter: “Is there such thing as estimation error on sample variance?” The answer is yes. Just as we saw earlier that the sample mean has a distribution, so does the sample variance. If you have different samples, you will get different variances and those will form a distribution. We are thus faced with exactly the same inference question as we were with the sample mean. How do we go about using the sample variance to estimate the actual variance? 

I will write a lot more about inference in the future, but for now suffice it to say: the biggest mistake being made (and it is being made all the time), is to mistake the Continue reading "Uncertainty Wednesday: Sample Variance (Cont’d)"

The Importance of Sleep

It appears that some people genetically need only 4 hours of sleep per night. Well, I am not one of them. I also don’t seem to be needing less sleep as I am getting older. Instead, I have come to appreciate how critically important sleep is for me. Looking back, many of my worst decisions, as well as my worst behavior, correlate with periods of insufficient sleep (or excessive jet lag for that matter).

The particular effects of not sleeping enough for me areL I become short tempered and easily irritable. That’s not good as an investor, colleague, friend, father and husband. I have spent a fair bit of time over the last couple of years working on my equanimity. I have made good progress on that generally, but take away my sleep and I will easily revert.

I fully realize that sleeping enough has become a privilege, as many Continue reading "The Importance of Sleep"

Uncertainty Wednesday: Sample Variance

Towards the end of last year in Uncertainty Wednesday, I wrote a post about suppressed volatility and gave an example. I ended the write up with:

if we simply estimate the volatility of a process from the observed sample variance, we may be wildly underestimating potential future variance 

This turns out to be true not just for cases of “suppressed volatility” but much more broadly. For any fat tailed distribution, the sample variance will underestimate the true variance. Mistaking the sample variance for the actual variance is the same error as mistaking the sample mean for the actual mean. The sample mean has a distribution and the sample variance has a distribution. Whether or not they are an unbiased estimator for the true values depends on the characteristics of the process.

Consider objects colliding with earth. Small objects strike earth with relatively high frequency. But how should we use a Continue reading "Uncertainty Wednesday: Sample Variance"

Martin Luther King Jr Supported Basic Income

We have the USV office closed today in honor of Martin Luther King Jr. day. In his book “Where Do We Go From Here: Chaos or Community” he wrote:

In addition to the absence of coordination and sufficiency, the [social] programs of the past all have another common failing — they are indirect. Each seeks to solve poverty by first solving something else.

I’m now convinced that the simplest approach will prove to be the most effective — the solution to poverty is to abolish it directly by a now widely discussed measure: the guaranteed income.

I strongly recommend reading a longer excerpt from the book, which shows just how visionary MLK was. He wrote the book in 1967, the year that I was born. Since then we have made extraordinary progress in the productive capacity of the economy. Put differently, we can afford a basic income more easily than ever before.

Continue reading "Martin Luther King Jr Supported Basic Income"

Martin Luther King Jr Supported Basic Income

We have the USV office closed today in honor of Martin Luther King Jr. day. In his book “Where Do We Go From Here: Chaos or Community” he wrote:

In addition to the absence of coordination and sufficiency, the [social] programs of the past all have another common failing — they are indirect. Each seeks to solve poverty by first solving something else.

I’m now convinced that the simplest approach will prove to be the most effective — the solution to poverty is to abolish it directly by a now widely discussed measure: the guaranteed income.

I strongly recommend reading a longer excerpt from the book, which shows just how visionary MLK was. He wrote the book in 1967, the year that I was born. Since then we have made extraordinary progress in the productive capacity of the economy. Put differently, we can afford a basic income more easily than ever before.

Continue reading "Martin Luther King Jr Supported Basic Income"

A SIM Switch Account Takeover (Mine)

It happened on the second to last day of a wonderful family trip to Southeast Asia. I looked at my mobile phone in the morning and it had an alert that said something like “SIM not recognized.” I should have probably figured out something was remiss right then, but instead I simply assumed that my phone had tried to register on an unsupported network. As I was sitting down for breakfast suddenly 3 emails arrived in rapid succession that made me realize I was being hacked (time order is from bottom to top):

image

Argh! Clearly someone had gotten my SMS messages to go to them instead and used it to hack my old yahoo email account. They quickly changed the password to lock me out and removed my alternate email.

From there I figured their next stop would be Twitter. That’s one of the few services where I used that

image
Continue reading "A SIM Switch Account Takeover (Mine)"

Meltdown and Spectre are Good … for Innovation

So I am still away on family vacation and following a self-imposed online diet, but even then it has been impossible to ignore the monster sized vulnerabilities disclosed today known as Meltdown and Spectre. And just to make sure nobody misreads my post title, these are bad. Downright ugly. They are pervasive, exploitable and a real longterm fix will likely require new hardware (one ore more extra hardware bits in the CPU). So how can I possibly claim they are good? Here are four different ways I think these vulnerabilities can give an important boost to innovation.

1. Faster Adoption of (Real) Cloud Computing

One might think that Meltdown and Spectre are terrible for cloud computing as they break through all memory isolation, so that an attacker can see everything that’s in memory on a physical machine (across all the virtual machines). But I believe the opposite will Continue reading "Meltdown and Spectre are Good … for Innovation"

Blogging, Twitter and Email Break

As 2017 draws to a close I will be taking a break from blogging, twitter and (for the most part) email until well into January. I have a great set of book recommendations – far too many to get through for quite some time – and I will be traveling with Susan and our children. Given all the craziness of this year in politics, climate change, technology (crypto currencies) and more, I look forward to getting some distance from it all and spending time with family. I feel incredibly fortunate to be able to do this and wish everyone all the best for 2018. 

Uber’s Greatest Trick Caught

In 2015 I wrote a blog post titled “Uber’s Greatest Trick Revealed” in which I argued that Uber’s success was the result of providing a transportation service. This was and is exactly what consumers wanted but is not a neutral platform or marketplace. It would appear that regulators have finally caught up with this with the European Union’s top court ruling yesterday that Uber is in fact a transportation service.

Now we will hopefully enter a new phase in which regulators figure out how to get consumers the benefits of on demand, app based dispatch, including the massive expansion of capacity, but still deal with issues such as safety, congestion, driver’s rights, etc. And yes some local regulators were and are captive to incumbent taxi companies but that doesn’t mean there are not enlightened ones to be found who will come up with the right rules that can then Continue reading "Uber’s Greatest Trick Caught"

Uncertainty Wednesday: Suppressed Volatility (Cont’d)

This will be the last Uncertainty Wednesday for 2017 as I am about to go away on vacation. In the last post I had introduced the idea that sometimes when volatility is suppressed it comes back to bite us. I wanted to have a really simple model for demonstrating that and so I wrote some Python code to make a 50:50 coin toss and depending on the result either increment or decrement a value by +/- 2 (I set the initial value to 100). Here is a plot of a sample run:

image

Now to suppress volatility, I modified the program so that it would increment or decrement the value by +/- 1 instead, i.e. half the original change. I then added the suppressed +/- 1 (the other half of the change) into a “buffer” – accumulating +1 in a positive buffer and -1 in a negative buffer. I then gave

image
image
Continue reading "Uncertainty Wednesday: Suppressed Volatility (Cont’d)"

Looking for Book Recommendations

We are about to go away on a three week trip. Long trips have been a wonderful aspect of homeschooling our children. We are fortunate to have the means to show them different parts of the world and this year we are going to Southeast Asia, which will also be new to Susan and myself. 

During our travels I also tend to detox from the Internet and find time to read. I already have a couple of books with me about the region we are going to, but am looking for a few more recommendations (I will also be reading this month’s USV book club selection, “The Sellout”).

I would love to hear from people: what is the best book you have read in the last 12 months and why?

Doug Jones Victory: Trump is Vulnerable

At our house we were all refreshing our computers furiously starting at 8pm with ever increasing excitement as the evening progressed. We were absolutely thrilled when Doug Jones victory was certain. If Doug Jones can win in Alabama, the state where Trump had his biggest victory, after Trump endorsed his opponent, well then Trump too can be defeated.

I have come to think of Trump’s candidacy and presidency as the last hurrah of a past that we need to leave behind for good. Trump, and the money behind him, waged a symbolic campaign of divisiveness that has continued with him in in office. It has been a mistake so far to try and defeat him with logical arguments. A while back I suggested that Take The Knee might be the right symbolic counter, but I was wrong about that, despite racism being one of the divisions Trump has exploited. 

Continue reading "Doug Jones Victory: Trump is Vulnerable"