Ygthkb

Is it common to refer to someone as "Prof. Dr. [LastName]"?

What does it mean when an external ID field follows a DML Statement?

How should I ship cards?

Can I legally make a website about boycotting a certain company?

Was Opportunity's last message to Earth "My battery is low and it's getting dark"?

Why do single electrical receptacles exist?

Isn't a semicolon (';') needed after a function declaration in C++?

Why would you use 2 alternate layout buttons instead of 1, when only one can be selected at once

Is there any danger of my neighbor having my wife's signature?

How to typeset a small black square as a binary operator?

Question: "Are you hungry?" Answer: "I feel like eating."

Variance of sine and cosine of a random variable

What is the name of this perspective and how is it constructed?

Why is quixotic not Quixotic (a proper adjective)?

Multiple null checks in Java 8

Do these large-scale, human power-plant-tending robots from the Matrix movies have a name, in-universe or out?

Badly designed reimbursement form. What does that say about the company?

What does "don't have a baby" imply or mean in this sentence?

If I have Haste cast on me, does it reduce the casting time for my spells that normally take more than a turn to cast?

SQL Server 2017 crashes when backing up because filepath is wrong

Did the characters in Moving Pictures not know about cameras like Twoflower's?

Taking an academic pseudonym?

What does an unprocessed RAW file look like?

For the Circle of Spores druid's Halo of Spores feature, is your reaction used regardless of whether the other creature succeeds on the saving throw?

Postgresql: How to store high-dimensional ( N > 100) vectors and index for fast lookup by cosine similarity?

Quick nearest neighbor search in the 150-dimensional spaceIs there any index with O(1) complexity for lookup in PostgreSQL?Hard and Fast rule for include columns in indexKey Lookup and Full-text indexUsing an index for both uniqueness and fast lookup?PostgreSQL FTS and Trigram-similarity Query OptimizationBest index for similarity functionPostgreSQL - How does multicolumn B-Tree index work with order by on 1st column and IN lookup for 2nd?Using SP-Gist or Gist(cube) index to implement kdtree for high dimensional data in PostgresqlHow can I index a text[] field in postgresql?Index for split string. PostgreSQL 10

0

I am trying to store vectors for word/doc embeddings in a postgresql table, and want to be able to quickly pull the N rows with highest cosine similarity to a given query vector. The vectors I'm working with are numpy.arrays of floats with length 100 <= L <= 1000.

I looked into the cube module for similarity search, but it is limited to vectors with <= 100 dimensions. The embeddings I am using will result in vectors that are 100-dimensions minimum and often much higher (depending on settings when training word2vec/doc2vec models).

What is the most efficient way to store large dimensional vectors (numpy float arrays) in postgres, and perform quick lookup based on cosine similarity (or other vector similarity metrics)?

postgresql index array dimension

share|improve this question

edited 3 mins ago

J. Taylor

asked 4 hours ago

J. TaylorJ. Taylor

132213

add a comment | 

0

I am trying to store vectors for word/doc embeddings in a postgresql table, and want to be able to quickly pull the N rows with highest cosine similarity to a given query vector. The vectors I'm working with are numpy.arrays of floats with length 100 <= L <= 1000.

I looked into the cube module for similarity search, but it is limited to vectors with <= 100 dimensions. The embeddings I am using will result in vectors that are 100-dimensions minimum and often much higher (depending on settings when training word2vec/doc2vec models).

What is the most efficient way to store large dimensional vectors (numpy float arrays) in postgres, and perform quick lookup based on cosine similarity (or other vector similarity metrics)?

postgresql index array dimension

share|improve this question

edited 3 mins ago

J. Taylor

asked 4 hours ago

J. TaylorJ. Taylor

132213

add a comment | 

I am trying to store vectors for word/doc embeddings in a postgresql table, and want to be able to quickly pull the N rows with highest cosine similarity to a given query vector. The vectors I'm working with are numpy.arrays of floats with length 100 <= L <= 1000.

I looked into the cube module for similarity search, but it is limited to vectors with <= 100 dimensions. The embeddings I am using will result in vectors that are 100-dimensions minimum and often much higher (depending on settings when training word2vec/doc2vec models).

What is the most efficient way to store large dimensional vectors (numpy float arrays) in postgres, and perform quick lookup based on cosine similarity (or other vector similarity metrics)?

postgresql index array dimension

share|improve this question

edited 3 mins ago

J. Taylor

asked 4 hours ago

J. TaylorJ. Taylor

132213

share|improve this question

edited 3 mins ago

J. Taylor

asked 4 hours ago

J. TaylorJ. Taylor

132213

share|improve this question

edited 3 mins ago

J. Taylor

asked 4 hours ago

J. TaylorJ. Taylor

132213

asked 4 hours ago

J. TaylorJ. Taylor

132213

asked 4 hours ago

J. TaylorJ. Taylor

132213