Vector Podcast: Recent Episodes

View Details

00:00 Intro

00:30 Greets for Doug

01:46 Apache Solr and stuff

03:08 Hello LTR project

04:42 Secret sauce of Doug's continuous blogging

08:50 SearchArray

13:22 Running complex ML experiments

17:29 Efficient search orgs

22:58 Writing a book on search and AI

Show notes:

• Doug's talk on Learning To Rank at Reddit delivered at the Berlin Buzzwords 2024 conference: https://www.youtube.com/watch?v=gUtF1gyHsSM

• Hello LTR: https://github.com/o19s/hello-ltr

• Lexical search for pandas with SearchArray: https://github.com/softwaredoug/searcharray

• https://softwaredoug.com/

• What AI Engineers Should Know about Search: https://softwaredoug.com/blog/2024/06/25/what-ai-engineers-need-to-know-search

• AI Powered Search: https://www.manning.com/books/ai-powered-search

• Quepid: https://github.com/o19s/quepid

• Branching out in your ML / search experiments: https://dvc.org/doc/use-cases

• Doug on Twitter: https://x.com/softwaredoug

• Doug on LinkedIn: https://www.linkedin.com/in/softwaredoug/

View Details

00:00 Intro

00:21 Guest Introduction: Eric Pugh

03:00 Eric's story in search and the evolution of search technology

7:27 Quepid: Improving Search Relevancy

10:08 When to use Quepid's

14:53 Flash back to Apache Solr 1.4 and the book (of which Eric is one author)

17:49 Quepid Demo and Future Enhancements

23:57 Real-Time Query Doc Pairs with WebSockets

24:16 Integrating Quepid with Search Engines

25:57 Introducing LLM-Based Judgments

28:05 Scaling Up Judgments with AI

28:48 Data Science Notebooks in Quepid

33:23 Custom Scoring in Quepid

39:23 API and Developer Tools

42:17 The Future of Search and Personal Reflections

Show notes:

• Hosted Quepid: https://app.quepid.com/

• Ragas: Evaluation framework for your Retrieval Augmented Generation (RAG) pipelines https://github.com/explodinggradients...

• Why Quepid: https://quepid.com/why-quepid/

• Quepid on Github: https://github.com/o19s/quepid

View Details

00:00 Intro

01:54 Reflection on the past year in AI

08:08 Reader LLM (and RAG)

12:36 Does it need fine-tuning to a domain?

14:20 How LLMs can lie

17:32 What if data isn't perfect

21:21 SWIRL's secret sauce with Reader LLM

23:55 Feedback loop

26:14 Some surprising client perspective

31:17 How Gen AI can change communication interfaces

34:11 Call-out to the Community

View Details

00:00 Intro

00:42 Louis's background

05:39 From Facebook to Rockset

07:41 Embeddings prior to deep learning / LLM era

12:35 What's Rockset as a product

15:27 Use cases

18:04 RocksDB as part of Rockset

20:33 AI capabilities: ANN index, hybrid search

25:11 Types of hybrid search

28:05 Can one learn the alpha?

30:03 Louis's prediction of the future of vector search

33:55 RAG and other AI capabilities

41:46 Call out to the Vector Search community

46:16 Vector Databases vs Databases

49:16 Question of WHY

View Details

Topics:

00:00 Intro - how do you like our new design?

00:52 Greets

01:55 Saurabh's background

03:04 Resume Matcher: 4.5K stars, 800 community members, 1.5K forks

04:11 How did you grow the project?

05:42 Target audience and how to use Resume Matcher

09:00 How did you attract so many contributors?

12:47 Architecture aspects

15:10 Cloud or not

16:12 Challenges in maintaining OS projects

17:56 Developer marketing with Swirl AI Connect

21:13 What you (listener) can help with

22:52 What drives you?

Show notes:

• Resume Matcher: https://github.com/srbhr/Resume-Matcher

website: https://resumematcher.fyi/

• Ultimate CV by Martin John Yate: https://www.amazon.com/Ultimate-CV-Cr...

• fastembed: https://github.com/qdrant/fastembed

• Swirl: https://github.com/swirlai/swirl-search

View Details

Topics: 00:00 Intro 00:22 Quick demo of SWIRL on the summary transcript of this episode 01:29 Sid’s background 08:50 Enterprise vs Federated search 17:48 How vector search covers for missing folksonomy in enterprise data 26:07 Relevancy from vector search standpoint 31:58 How ChatGPT improves programmer’s productivity 32:57 Demo! 45:23 Google PSE 53:10 Ideal user of SWIRL 57:22 Where SWIRL sits architecturally 1:01:46 How to evolve SWIRL with domain expertise 1:04:59 Reasons to go open source 1:10:54 How SWIRL and Sid interact with ChatGPT 1:23:22 The magical question of WHY 1:27:58 Sid’s announcements to the community

YouTube version: https://www.youtube.com/watch?v=vhQ5LM5pK_Y

Design by Saurabh Rai: https://twitter.com/_srbhr_ Check out his Resume Matcher project: https://www.resumematcher.fyi/

View Details

Topics:

00:00 Intro

02:20 Atita’s path into search engineering

09:00 When it’s time to contribute to open source

12:08 Taking management role vs software development

14:36 Knowing what you like (and coming up with a Solr course)

19:16 Read the source code (and cook)

23:32 Open Bistro Innovations Lab and moving to Germany

26:04 Affinity to Search world and working as a Search Relevance Consultant

28:39 Bringing vector search to Chorus and Querqy

34:09 What Atita learnt from Eric Pugh’s approach to improving Quepid

36:53 Making vector search with Solr & Elasticsearch accessible through tooling and documentation

41:09 Demystifying data embedding for clients (and for Java based search engines)

43:10 Shifting away from generic to domain-specific in search+vector saga

46:06 Hybrid search: where it will be useful to combine keyword with semantic search

50:53 Choosing between new vector DBs and “old” keyword engines

58:35 Women of Search

1:14:03 Important (and friendly) People of Open Source

1:22:38 Reinforcement learning applied to our careers

1:26:57 The magical question of WHY

1:29:26 Announcements

See show notes on YouTube: https://www.youtube.com/watch?v=BVM6TUSfn3E

View Details

Topics:

00:00 Intro

01:54 Things Connor learnt in the past year that changed his perception of Vector Search

02:42 Is search becoming conversational?

05:46 Connor asks Dmitry: How Large Language Models will change Search?

08:39 Vector Search Pyramid

09:53 Large models, data, Form vs Meaning and octopus underneath the ocean

13:25 Examples of getting help from ChatGPT and how it compares to web search today

18:32 Classical search engines with URLs for verification vs ChatGPT-style answers

20:15 Hybrid search: keywords + semantic retrieval

23:12 Connor asks Dmitry about his experience with sparse retrieval

28:08 SPLADE vectors

34:10 OOD-DiskANN: handling the out-of-distribution queries, and nuances of sparse vs dense indexing and search

39:54 Ways to debug a query case in dense retrieval (spoiler: it is a challenge!)

44:47 Intricacies of teaching ML models to understand your data and re-vectorization

49:23 Local IDF vs global IDF and how dense search can approach this issue

54:00 Realtime index

59:01 Natural language to SQL

1:04:47 Turning text into a causal DAG

1:10:41 Engineering and Research as two highly intelligent disciplines

1:18:34 Podcast search

1:25:24 Ref2Vec for recommender systems

1:29:48 Announcements

For Show Notes, please check out the YouTube episode below.

This episode on YouTube: https://www.youtube.com/watch?v=2Q-7taLZ374

Podcast design: Saurabh Rai: https://twitter.com/srvbhr

View Details

Toloka’s support for Academia: grants and educator partnerships

https://toloka.ai/collaboration-with-educators-form

https://toloka.ai/research-grants-form

These are pages leading to them:

https://toloka.ai/academy/education-partnerships

https://toloka.ai/grants

Topics:

00:00 Intro

01:25 Jenny’s path from graduating in ML to a Data Advocate role

07:50 What goes into the labeling process with Toloka

11:27 How to prepare data for labeling and design tasks

16:01 Jenny’s take on why Relevancy needs more data in addition to clicks in Search

18:23 Dmitry plays the Devil’s Advocate for a moment

22:41 Implicit signals vs user behavior and offline A/B testing

26:54 Dmitry goes back to advocating for good search practices

27:42 Flower search as a concrete example of labeling for relevancy

39:12 NDCG, ERR as ranking quality metrics

44:27 Cross-annotator agreement, perfect list for NDCG and Aggregations

47:17 On measuring and ensuring the quality of annotators with honeypots

54:48 Deep-dive into aggregations

59:55 Bias in data, SERP, labeling and A/B tests

1:16:10 Is unbiased data attainable?

1:23:20 Announcements

This episode on YouTube: https://youtu.be/Xsw9vPFqGf4

Podcast design: Saurabh Rai: https://twitter.com/srvbhr

View Details

00:00 Introduction

01:11 Yaniv’s background and intro to Searchium & GSI

04:12 Ways to consume the APU acceleration for vector search

05:39 Power consumption dimension in vector search

7:40 Place of the platform in terms of applications, use cases and developer experience

12:06 Advantages of APU Vector Search Plugins for Elasticsearch and OpenSearch compared to their own implementations

17:54 Everyone needs to save: the economic profile of the APU solution

20:51 Features and ANN algorithms in the solution

24:23 Consumers most interested in dedicated hardware for vector search vs SaaS

27:08 Vector Database or a relevance oriented application?

33:51 Where to go with vector search?

42:38 How Vector Search fits into Search

48:58 Role of the human in the AI loop

58:05 The missing bit in the AI/ML/Search space

1:06:37 Magical WHY question

1:09:54 Announcements

• Searchium vector search: https://searchium.ai/

• Dr. Avidan Akerib, founder behind the APU technology: https://www.linkedin.com/in/avidan-akerib-phd-bbb35b12/

• OpenSearch benchmark for performance tuning: https://betterprogramming.pub/tired-of-troubleshooting-idle-search-resources-use-opensearch-benchmark-for-performance-tuning-d4277c9f724

• APU KNN plugin for OpenSearch: https://towardsdatascience.com/bolster-opensearch-performance-with-5-simple-steps-ca7d21234f6b

• Multilingual and Multimodal Search with Hardware Acceleration: https://blog.muves.io/multilingual-and-multimodal-vector-search-with-hardware-acceleration-2091a825de78

• Muves talk at Berlin Buzzwords, where we have utilized GSI APU: https://blog.muves.io/muves-at-berlin-buzzwords-2022-3150eef01c4

• Not All Vector Databases are made equal: https://towardsdatascience.com/milvus-pinecone-vespa-weaviate-vald-gsi-what-unites-these-buzz-words-and-what-makes-each-9c65a3bd0696

Episode on YouTube: https://youtu.be/EerdWRPuqd4

Podcast design: Saurabh Rai: https://twitter.com/srvbhr

View Details

Topics:

00:00 Intro

01:30 Doug’s story in Search

04:55 How Quepid came about

10:57 Relevance as product at Shopify: challenge, process, tools, evaluation

15:36 Search abandonment in Ecommerce

21:30 Rigor in A/B testing

23:53 Turn user intent and content meaning into tokens, not words into tokens

32:11 Use case for vector search in Maps. What about search in other domains?

38:05 Expanding on dense approaches

40:52 Sparse, dense, hybrid anyone?

48:18 Role of HNSW, scalability and new vector databases vs Elasticsearch / Solr dense search

52:12 Doug’s advice to vector database makers

58:19 Learning to Rank: how to start, how to collect data with active learning, what are the ML methods and a mindset

1:12:10 Blending search and recommendation

1:16:08 Search engineer role and key ingredients of managing search projects today

1:20:34 What does a Product Manager do on a Search team?

1:26:50 The magical question of WHY

1:29:08 Doug’s announcements

Show notes:

Doug’s course: https://www.getsphere.com/ml-engineering/ml-powered-search?source=Instructor-Other-070922-vector-pod

Upcoming book: https://www.manning.com/books/ai-powered-search?aaid=1&abid=e47ada24&chan=aips

Doug’s post in Shopify’s blog “Search at Shopify—Range in Data and Engineering is the Future”: https://shopify.engineering/search-at-shopify

Doug’s own blog: https://softwaredoug.com/

Using Bayesian optimization for Elasticsearch relevance: https://www.youtube.com/watch?v=yDcYi-ANJwE&t=1s

Hello LTR: https://github.com/o19s/hello-ltr

Vector Databases: https://towardsdatascience.com/milvus-pinecone-vespa-weaviate-vald-gsi-what-unites-these-buzz-words-and-what-makes-each-9c65a3bd0696

Research: Search abandonment has a lasting impact on brand loyalty: https://cloud.google.com/blog/topics/retail/search-abandonment-impacts-retail-sales-brand-loyalty

Quepid: https://quepid.com/

Podcast design: Saurabh Rai [https://twitter.com/srvbhr]

View Details

Topics:

00:00 Introduction

01:12 Malte’s background

07:58 NLP crossing paths with Search

11:20 Product discovery: early stage repetitive use cases pre-dating Haystack

16:25 Acyclic directed graph for modeling a complex search pipeline

18:22 Early integrations with Vector Databases

20:09 Aha!-use case in Haystack

23:23 Capabilities of Haystack today

30:11 Deepset Cloud: end-to-end deployment, experiment tracking, observability, evaluation, debugging and communicating with stakeholders

39:00 Examples of value for the end-users of Deepset Cloud

46:00 Success metrics

50:35 Where Haystack is taking us beyond MLOps for search experimentation

57:13 Haystack as a smart assistant to guide experiments

1:02:49 Multimodality

1:05:53 Future of the Vector Search / NLP field: large language models

1:15:13 Incorporating knowledge into Language Models & an Open NLP Meetup on this topic

1:16:25 The magical question of WHY

1:23:47 Announcements from Malte

Show notes:

• Haystack: https://github.com/deepset-ai/haystack/

• Deepset Cloud: https://www.deepset.ai/deepset-cloud

• Tutorial: Build Your First QA System: https://haystack.deepset.ai/tutorials/v0.5.0/first-qa-system

• Open NLP Meetup on Sep 29th (Nils Reimers talking about “Incorporating New Knowledge Into LMs”): https://www.meetup.com/open-nlp-meetup/events/287159377/

• Atlas Paper (Few shot learning with retrieval augmented large language models): https://arxiv.org/abs/2208.03299

• Tweet from Patrick Lewis: https://twitter.com/PSH_Lewis/status/1556642671569125378

• Zero click search: https://www.searchmetrics.com/glossary/zero-click-searches/

Very large LMs:

• 540B PaLM by Google: https://lnkd.in/eajsjCMr

• 11B Atlas by Meta: https://lnkd.in/eENzNkrG

• 20B AlexaTM by Amazon: https://lnkd.in/eyBaZDTy

• Players in Vector Search: https://www.youtube.com/watch?v=8IOpgmXf5r8 https://dmitry-kan.medium.com/players-in-vector-search-video-2fd390d00d6

• Click Residual: A Query Success Metric: https://observer.wunderwood.org/2022/08/08/click-residual-a-query-success-metric/

• Tutorials and papers around incorporating Knowledge into Language Models: https://cs.stanford.edu/people/cgzhu/

Podcast design: Saurabh Rai https://twitter.com/srvbhr

View Details

00:00 Introduction

01:10 Max's deep experience in search and how he transitioned from structured data

08:28 Query-term dependence problem and Max's perception of the Vector Search field

12:46 Is vector search a solution looking for a problem?

20:16 How to move embeddings computation from GPU to CPU and retain GPU latency?

27:51 Plug-in neural model into Java? Example with a Hugging Face model

33:02 Web-server Mighty and its philosophy

35:33 How Mighty compares to in-DB embedding layer, like Weavite or Vespa

39:40 The importance of fault-tolerance in search backends

43:31 Unit economics of Mighty

50:18 Mighty distribution and supported operating systems

54:57 The secret sauce behind Mighty's insane fast-ness

59:48 What a customer is paying for when buying Mighty

1:01:45 How will Max track the usage of Mighty: is it commercial or research use?

1:04:39 Role of Open Source Community to grow business

1:10:58 Max's vision for Mighty connectors to popular vector databases

1:18:09 What tooling is missing beyond Mighty in vector search pipelines

1:22:34 Fine-tuning models, metric learning and Max's call for partnerships

1:26:37 MLOps perspective of neural pipelines and Mighty's role in it

1:30:04 Mighty vs AWS Inferentia vs Hugging Face Infinity

1:35:50 What's left in ML for those who are not into Python

1:40:50 The philosophical (and magical) question of WHY

1:48:15 Announcements from Max

25% discount for the first year of using Mighty in your great product / project with promo code VECTOR:

https://bit.ly/3QekTWE

Show notes:

• Max's blog about BERT and search relevance: https://opensourceconnections.com/blog/2019/11/05/understanding-bert-and-search-relevance/

• Case study and unit economics of Mighty: https://max.io/blog/encoding-the-federal-register.html

• Not All Vector Databases Are Made Equal: https://towardsdatascience.com/milvus-pinecone-vespa-weaviate-vald-gsi-what-unites-these-buzz-words-and-what-makes-each-9c65a3bd0696

Watch on YouTube: https://youtu.be/LnF4hbl1cE4

View Details

Vector Podcast Live

Topics:

00:00 Kick-off introducing co:rise study platform

03:03 Grant’s background

04:58 Principle of 3 C’s in the life of a CTO: Code, Conferences and Customers

07:16 Principle of 3 C’s in the Search Engine development: Content, Collaboration and Context

11:51 Balance between manual tuning in pursuit to learn and Machine Learning

15:42 How to nurture intuition in building search engine algorithms

18:51 How to change the approach of organizations to true experimentation

23:17 Where should one start in approaching the data (like click logs) for developing a search engine

29:36 How to measure the success of your search engine

33:50 The role of manual query rating to improve search result relevancy

36:56 What are the available datasets, tools and algorithms, that allow us to build a search engine?

41:56 Vector search and its role in broad search engine development and how the profession is shaping up

49:01 The magical question of WHY: what motivates Grant to stay in the space

52:09 Announcement from Grant: course discount code DGSEARCH10

54:55 Questions from the audience

Show notes:

• Grant’s interview at Berlin Buzzwords 2016: https://www.youtube.com/watch?v=Y13gZM5EGdc

• “BM25 is so Yesterday: Modern Techniques for Better Search”: https://www.youtube.com/watch?v=CRZfc9lj7Po

• “Taming text” - book co-authored by Grant: https://www.manning.com/books/taming-text

• Search Fundamentals course - https://corise.com/course/search-fundamentals

• Search with ML course - https://corise.com/course/search-with-machine-learning

• Click Models for Web Search: https://github.com/markovi/PyClick

• Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing, book by Ron Kohavi et al: https://www.amazon.com/Trustworthy-Online-Controlled-Experiments-Practical-ebook/dp/B0845Y3DJV

• Quepid, open source tool and free service for query rating and relevancy tuning: https://quepid.com/

• Grant’s talk in 2013 where he discussed the need of a vector field in Lucene and Solr: https://www.youtube.com/watch?v=dCCqauwMWFE

• CLIP model for multimodal search: https://openai.com/blog/clip/

• Demo of multimodal search with CLIP: https://blog.muves.io/multilingual-and-multimodal-vector-search-with-hardware-acceleration-2091a825de78

• Learning to Boost: https://www.youtube.com/watch?v=af1dyamySCs

• Dmitry’s Medium List on Vector Search: https://medium.com/@dmitry-kan/list/vector-search-e9b564d14274

View Details

Topics:

00:00 Kick-off by Judy Zhu

01:33 Introduction by Dmitry Kan and his bio!

03:03 Daniel’s background

04:46 “Science is the difference between instinct and strategy”

07:41 Search as a personal learning experience

11:53 Why do we need Machine Learning in Search, or can we use manually curated features?

16:47 Swimming up-stream from relevancy: query / content understanding and where to start?

23:49 Rule-based vs Machine Learning approaches to Query Understanding: Pareto principle

29:05 How content understanding can significantly improve your search engine experience

32:02 Available datasets, tools and algorithms to train models for content understanding

38:20 Daniel’s take on the role of vector search in modern search engine design as the path to language of users

45:17 Mystical question of WHY: what drives Daniel in the search space today

49:50 Announcements from Daniel

51:15 Questions from the audience

Show notes:

What is Content Understanding?. Content understanding is the foundation… | by Daniel Tunkelang | Content Understanding | Medium

Query Understanding: An Introduction | by Daniel Tunkelang | Query Understanding

Science as Strategy (https://www.youtube.com/watch?v=dftt6Yqgnuw)

Search Fundamentals course - https://corise.com/course/search-fundamentals

Search with ML course - https://corise.com/course/search-with-machine-learning

Books:

Faceted Search, by Daniel Tunkelang: https://www.amazon.com/Synthesis-Lectures-Information-Concepts-Retrieval/dp/1598299999

Modern Information Retrieval: The Concepts and Technology Behind Search, by Ricardo Baeza-Yates: https://www.amazon.com/Modern-Information-Retrieval-Concepts-Technology/dp/0321416910/ref=sr11?qid=1653144684&refinements=p_27%3ARicardo+Baeza-Yates&s=books&sr=1-1

Introduction to Information Retrieval, by Chris Manning: https://www.amazon.com/Introduction-Information-Retrieval-Christopher-Manning/dp/0521865719/ref=sr1fkmr0_1?crid=2GIR19OTZ8QFJ&keywords=chris+manning+information+retrieval&qid=1653144967&s=books&sprefix=chris+manning+information+retrieval%2Cstripbooks-intl-ship%2C141&sr=1-1-fkmr0

Query Understanding for Search Engines, by Yi Chang and Hongbo Deng: https://www.amazon.com/Understanding-Search-Engines-Information-Retrieval/dp/3030583333

View Details

Topics:

00:00 Intro

01:03 Yusuf’s background

03:00 Multimodal search in tech and humans

08:53 CLIP: discovering hidden semantics

13:02 Where to start to apply metric learning in practice. AutoEncoder architecture included!

19:00 Unpacking it further: what is metric learning and the difference with deep metric learning?

28:50 How Deep Learning allowed us to transition from pixels to meaning in the images

32:05 Increasing efficiency: vector compression and quantization aspects

34:25 Yusuf gives a practical use-case with Conversational AI of where metric learning can prove to be useful. And tools!

40:59 A few words on how the podcast is made :) Yusuf’s explanation of how Gmail smart reply feature works internally

51:19 Metric learning helps us learn the best vector representation for the given task

52:16 Metric learning shines in data scarce regimes. Positive impact on the planet

58:30 Yusuf’s motivation to work in the space of vector search, Qdrant, deep learning and metric learning — the question of Why

1:05:02 Announcements from Yusuf

• Join discussions at Discord: https://discord.qdrant.tech

• Yusuf's Medium: https://medium.com/@yusufsarigoz and LinkedIn: https://www.linkedin.com/in/yusufsarigoz/

• GSOC 2022: TensorFlow Similarity - project led by Yusuf: https://docs.google.com/document/d/1fLDLwIhnwDUz3uUV8RyUZiOlmTN9Uzy5ZuvI8iDDFf8/edit#heading=h.zftd93u5hfnp

• Dmitry's Twitter: https://twitter.com/DmitryKan

Full Show Notes: https://www.youtube.com/watch?v=AU0O_6-EY6s

View Details

Topics:

00:00 Introduction

01:21 Jo Kristian’s background in Search / Recommendations since 2001 in Fast Search & Transfer (FAST)

03:16 Nice words about Trondheim

04:37 Role of NTNU in supplying search talent and having roots in FAST

05:33 History of Vespa from keyword search

09:00 Architecture of Vespa and programming language choice: C++ (content layer), Java (HTTP requests and search plugins) and Python (pyvespa)

13:45 How Python API enables evaluation of the latest ML models with Vespa and ONNX support

17:04 Tensor data structure in Vespa and its use cases

22:23 Multi-stage ranking pipeline use cases with Vespa

24:37 Optimizing your ranker for top 1. Bonus: cool search course mentioned!

30:18 Fascination of Query Understanding, ways to implement and its role in search UX

33:34 You need to have investment to get great results in search

35:30 Game-changing vector search in Vespa and impact of MS Marco Passage Ranking

38:44 User aspect of vector search algorithms

43:19 Approximate vs exact nearest neighbor search tradeoffs

47:58 Misconceptions in neural search

52:06 Ranking competitions, idea generation and BERT bi-encoder dream

56:19 Helping wider community through improving search over CORD-19 dataset

58:13 Multimodal search is where vector search shines

1:01:14 Power of building fully-fledged demos

1:04:47 How to combine vector search with sparse search: Reciprocal Rank Fusion

1:10:37 The philosophical WHY question: Jo Kristian’s drive in the search field

1:21:43 Announcement on the coming features from Vespa

• Jo Kristian’s Twitter: https://twitter.com/jobergum

• Dmitry’s Twitter: https://twitter.com/DmitryKan

For the Show Notes check: https://www.youtube.com/watch?v=UxEdoXtA9oM

View Details

Topics:

00:00 Intro

00:54 Amin’s background at Google Research and affinity to NLP and vector search field

05:28 Main focus areas of ZIR.AI in neural search

07:26 Does the company offer neural network training to clients? Other support provided with ranking and document format conversions

08:51 Usage of open source vs developing own tech

10:17 The core of ZIR.AI product

14:36 API support, communication protocols and P95/P99 SLAs, dedicated pools of encoders

17:13 Speeding up single node / single customer throughput and challenge of productionizing off the shelf models, like BERT

23:01 Distilling transformer models and why it can be out of reach of smaller companies

25:07 Techniques for data augmentation from Amin’s and Dmitry’s practice (key search team: margin loss)

30:03 Vector search algorithms used in ZIR.AI and the need for boolean logic in company’s client base

33:51 Dynamics of open source in vector search space and cloud players: Google, Amazon, Microsoft

36:03 Implementing a multilingual search with BM25 vs neural search and impact on business

38:56 Is vector search a hype similar to big data few years ago? Prediction for vector search algorithms influence relations databases

43:09 Is there a need to combine BM25 with neural search? Ideas from Amin and features offered in ZIR.AI product

51:31 Increasing the robustness of search — or simply making it to work

55:10 How will Search Engineer profession change with neural search in the game?

Get a $100 discount (first month free) for a 50mb plan, using the code VectorPodcast (no lock-in, you can cancel any time): https://zir-ai.com/signup/user

View Details

Topics:

00:00 Introduction

01:04 Yury’s background in laser physics, computer vision and startups

05:14 How Yury entered the field of nearest neighbor search and his impression of it

09:03 “Not all Small Worlds are Navigable”

10:10 Gentle introduction into the theory of Small World Navigable Graphs and related concepts

13:55 Further clarification on the input constraints for the NN search algorithm design

15:03 What did not work in NSW algorithm and how did Yury set up to invent new algorithm called HNSW

24:06 Collaboration with Leo Boytsov on integrating HNSW in nmslib

26:01 Differences between HNSW and NSW

27:55 Does algorithm always converge?

31:56 How FAISS’s implementation is different from the original HNSW

33:13 Could Yury predict that his algorithm would be implemented in so many frameworks and vector databases in languages like Go and Rust?

36:51 How our perception of high-dimensional spaces change compared to 3D?

38:30 ANN Benchmarks

41:33 Feeling proud of the invention and publication process during 2,5 years!

48:10 Yury’s effort to maintain HNSW and its GitHub community and the algorithm’s design principles

53:29 Dmitry’s ANN algorithm KANNDI, which uses HNSW as a building block

1:02:16 Java / Python Virtual Machines, profiling and benchmarking. “Your analysis of performance contradicts the profiler”

1:05:36 What are Yury’s hopes and goals for HNSW and role of symbolic filtering in ANN in general

1:13:05 The future of ANN field: search inside a neural network, graph ANN

1:15:14 Multistage ranking with graph based nearest neighbor search

1:18:18 Do we have the “best” ANN algorithm? How ANN algorithms influence each other

1:21:27 Yury’s plans on publishing his ideas

1:23:42 The intriguing question of Why

Show notes:

• HNSW library: https://github.com/nmslib/hnswlib/

• HNSW paper Malkov, Y. A., & Yashunin, D. A. (2018). Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. TPAMI, 42(4), 824-836. (arxiv:1603.09320)

• NSW paper Malkov, Y., Ponomarenko, A., Logvinov, A., & Krylov, V. (2014). Approximate nearest neighbor algorithm based on navigable small world graphs. Information Systems, 45, 61-68.

• Yury Lifshits’s paper: https://yury.name/papers/lifshits2009combinatorial.pdf

• Sergey Brin’s work in nearest neighbour search: GNAT - Geometric Near-neighbour Access Tree: CiteSeerX — Near neighbor search in large metric spaces

• Podcast with Leo Boytsov: https://rare-technologies.com/rrp-4-leo-boytsov-knn-search/

• Million-Scale ANN Benchmarks: http://ann-benchmarks.com/

• Billion Scale ANN Benchmarks: https://github.com/harsha-simhadri/big-ann-benchmarks

• FALCONN algorithm: https://github.com/falconn-lib/falconn

• Mentioned navigable small world papers:

Kleinberg, J. M. (2000). Navigation in a small world. Nature, 406(6798), 845-845.;

Boguna, M., Krioukov, D., & Claffy, K. C. (2009). Navigability of complex networks. Nature Physics, 5(1), 74-80.

View Details

Topics:

00:00 Intro

00:42 Joan's background

01:46 What attracted Joan's attention in Jina as a company and product?

04:39 Main area of focus for Joan in the product

05:46 How Open Source model works for Jina?

08:38 Deeper dive into Jina.AI as a product and technology stack

11:57 Does Jina fit the use cases of smaller / mid-size players with smaller amount of data?

13:45 KNN/ANN algorithms available in Jina

16:05 BigANN competition and BuddyPQ, increasing 12% in recall over FAISS

17:07 Does Jina support customers in model training? Finetuner

20:46 How does Jina framework compare to Vector Databases?

26:46 Jina's investment in user-friendly APIs

31:04 Applications of Jina beyond search engines, like question answering systems

33:20 How to bring bits of neural search into traditional keyword retrieval? Connection to model interpretability

41:14 Does Jina allow going multimodal, including images / audio etc?

46:03 The magical question of Why

55:20 Product announcement from Joan

Order your Jina swag https://docs.google.com/forms/d/e/1FAIpQLSedYVfqiwvdzWPX-blCpVu-tQoiFiUJQz2QnIHU1ggy1oyg/ Use this promo code: vectorPodcastxJinaAI

Show notes:

• Jina.AI: https://jina.ai/

• HNSW + PostgreSQL Indexer: GitHub - jina-ai/executor-hnsw-postgres: A production-ready, scalable Indexer for the Jina neural search framework, based on HNSW and PSQL

• pqlite: GitHub - jina-ai/pqlite: A fast embedded library for Approximate Nearest Neighbor Search integrated with the Jina ecosystem

• BuddyPQ: Billion-Scale Vector Search: Team Sisu and BuddyPQ | by Dmitry Kan | Big-ANN-Benchmarks | Nov, 2021 | Medium

• PaddlePaddle: GitHub - PaddlePaddle/Paddle: PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

• Jina Finetuner: Finetuner 0.3.1 documentation

• Not All Vector Databases Are Made Equal | by Dmitry Kan | Towards Data Science

• Fluent interface (method chaining): Fluent interfaces in Python | Florian Einfalt – Developer

• Sujit Pal’s blog: Salmon Run

• ByT5: Towards a token-free future with pre-trained byte-to-byte models https://arxiv.org/abs/2105.13626

Special thanks to Saurabh Rai for the Podcast Thumbnail: https://twitter.com/srbhr_ https://www.linkedin.com/in/srbh077/

View Details

Show notes:

• The ML Test Score: A Rubric for ML Production Readiness and Technical Debt Reduction https://research.google/pubs/pub46555/

• IEEE MLOps Standard for Ethical AI https://docs.google.com/document/d/1x...

• Qdrant: https://qdrant.tech/

• Elixir connector for Qdrant by Tom: https://github.com/tlack/exqdr

• Other 6 vector databases: https://towardsdatascience.com/milvus...

• ByT5: Towards a token-free future with pre-trained byte-to-byte models https://arxiv.org/abs/2105.13626

• Tantivy: https://github.com/quickwit-inc/tantivy

• Papers with code: https://paperswithcode.com/

View Details

Show notes:

• On the Measure of Intelligence by François Chollet - Part 1: Foundations (Paper Explained) (https://www.youtube.com/watch?v=3_qGr...)

• 2108.07258 On the Opportunities and Risks of Foundation Models

• 2005.11401 Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

• Negative Data Augmentation: https://arxiv.org/abs/2102.05113

• Beyond Accuracy: Behavioral Testing of NLP models with CheckList: 2005.04118 Beyond Accuracy: Behavioral Testing of NLP models with CheckList

• Symbolic AI vs Deep Learning battle https://www.technologyreview.com/2020...

• Dense Passage Retrieval for Open-Domain Question Answering https://arxiv.org/abs/2004.04906

• Data Augmentation Can Improve Robustness https://arxiv.org/abs/2111.05328

• Contrastive Loss Explained. Contrastive loss has been used recently… | by Brian Williams | Towards Data Science https://towardsdatascience.com/contra...

• Keras Code examples https://keras.io/examples/

• https://you.com/ -- new web search engine by Richard Socher

• The Book of Why: The New Science of Cause and Effect: Pearl, Judea, Mackenzie, Dana: 9780465097609: Amazon.com: Books https://www.amazon.com/Book-Why-Scien...

• Chelsea Finn: https://twitter.com/chelseabfinn

• Jeff Clune: https://twitter.com/jeffclune

• Michael Bronstein (Geometric Deep Learning): https://twitter.com/mmbronstein https://arxiv.org/abs/2104.13478

• Connor's Twitter: https://twitter.com/CShorten30

• Dmitry's Twitter: https://twitter.com/DmitryKan

View Details

Order your Milvus t-shirt / hoodie! https://milvus.typeform.com/to/IrnLAgui Thanks Filip for arranging.

Show notes:

• Milvus DB: https://milvus.io/

• Not All Vector Databases Are Made Equal: https://towardsdatascience.com/milvus...

• Milvus talk at Haystack: https://www.youtube.com/watch?v=MLSMs...

• BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models https://arxiv.org/abs/2104.08663

• End-to-End Environmental Sound Classification using a 1D Convolutional Neural Network: https://arxiv.org/abs/1904.08990

• What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models https://arxiv.org/abs/1907.13528

• NVIDIA Triton Inference Server: https://developer.nvidia.com/nvidia-t...

• Towhee -- ML / Embedding pipeline making steps before Milvus easier: https://github.com/towhee-io/towhee

• Being at the leading edge: http://paulgraham.com/startupideas.html

View Details

1. Layering problem: www.edge.org/conversation/sean_…-layers-of-reality

2. Podcast with Etienne Dilocker (SeMI Technologies Co-Founder & CTO): www.youtube.com/watch?v=6lkanzOqhDs

3. SOC2: linfordco.com/blog/soc-1-vs-soc-2-audit-reports/

4. Dmitry's post on 7 Vector Databases: towardsdatascience.com/milvus-pineco…-9c65a3bd0696

5. Billion-Scale ANN Challenge: big-ann-benchmarks.com/index.html

6. Weaviate Introduction: www.semi.technology/developers/weaviate/current/ Newsletter: www.semi.technology/newsletter/

7. Use case: Scalable Knowledge Graph Search for 60+ million academic papers with Weaviate: medium.com/keenious/knowledge-…aviate-7964657ec911

8. Bob's Twitter: twitter.com/bobvanluijt

9. Dmitry's Twitter: twitter.com/DmitryKan

10. Dmitry's tech blog: dmitry-kan.medium.com/

View Details

Show notes:

1. Pinecone 2.0: https://www.pinecone.io/learn/pinecon... It is GA and free: https://www.pinecone.io/learn/v2-pric...

2. Get your “Love Thy Nearest Neighbour” t-shirt :) shoot an email to greg@pinecone.io

3. Billion-Scale Approximate Nearest Neighbour Search Challenge: https://big-ann-benchmarks.com/index....

4. ANNOY: https://github.com/spotify/annoy

5. FAISS: https://github.com/facebookresearch/f...

6. HNSW: https://github.com/nmslib/hnswlib

7. “How Zero Results Are Killing Ecommerce Conversions” https://lucidworks.com/post/how-zero-...

8. Try out Pinecone vector DB: https://app.pinecone.io/

9. Twitter: https://twitter.com/Pinecone_io

10. LinkedIn: https://www.linkedin.com/company/pine...

11. Greg’s Twitter: https://twitter.com/grigoriy_kogan

12. Dmitry's Twitter: https://twitter.com/DmitryKan

Watch on YouTube: https://www.youtube.com/watch?v=jT3i7NLwJ8w