Code&Data Insights

Large Language Model (LLM) : 자연어 처리를 위해 방대한 양의 데이터로 학습된 인공지능 모델로 인간과 유사한 텍스트를 처리, 이해 및 생성하도록 설계된 고급 AI 시스템. A type of deep learning model trained on massive amounts of text data to learn the patterns and structures of language. They can perform language-related tasks, including text generation, translation, summarization, sentiment analysis, and more. - LLMs work by training on diverse language ..

AI hallucinations이란 ? : AI models이 생성한 incorrect하거나 misleading results. - 트레이닝 데이터가 충분하지 않거나 잘못된 가정, 데이터의 편향으로 인해 잘못된 패턴을 습득하여 잘못된 예측, hallucinations이 발생! - 특히, 의학적 진단, 금융 거래는 정확한 결과가 중요함! ==> hallucinations을 예방하거나 줄이고 정확한 결과를 얻기 위해 프롬프트 엔지니어링은 빠르고 쉬운 방법이 될 수 있다. AI hallucinations 왜 발생하는 가? (1) 생성 모델의 한계 : 생성적 AI 모델은 훈련 데이터에 기반하여 알고리즘적으로 작동함. - 관찰된 패턴을 기반으로 다음 단어나 시퀀스를 예측하는 것이 목표. - 진실을 검증하는 것에 ..

Zero-Shot Learning : 예시 없이 그냥 Prompt 입력 (one-shot learning : 하나의 예시만 제공) Zero-Shot Prompting : A method wherein models generate meaningful responses to prompts without prior training (ex) prompt : select the adjective in this sentence "Anita bakes the best cakes in the neighborhood." - output : "Best" => HOWEVER, 1-prompt로 desired output(원하는 결과)를 얻기는 힘듬!! User Feedback Loop : 계속되는 prompt(by user..

What Is Prompt Engineering? 프롬프트 엔지니어링은 AI 모델, 특히 LLM을 원하는 결과로 유도하기 위해 정교하게 설계된 질문이나 지시를 만드는 과정. 원하는 결과물을 얻기 위해 프롬프트를 최적화하는 기술 Prompt engineering is essential for enhancing AI-powered services and maximizing the output of existing generative AI tools. Powerful prompts lead to => Optimize the response of generative AI models => Power lies in the questions we ask => Write a prompt that is effectiv..

Attention - Contextual embedding => Transform each input embedding into a contextual imbedding => Model learns attention weights - Self-attention : allows to enhance the embedding of an input word by including information about its context - Encoder-Decoder Attention : Attention between words in the input sequence and words in the output sequence => how words from two sequences influence each ot..

Word Embeddings · Word Vectors - Simple approach : one-hot vectors => NOT represent word meanining => Similarity/distance between all one hot vectors is the same => Better approach : ‘Word Embeddings’! · Word2Vec : how likely is a word w likely to show up near another word? - Extract the learned weights as the word embeddings - Use as training set, readily available texts, no need for hand-label..

N-gram models : a probability distribution over sequence of events · Models the order of the events · Used when the past sequence of events is a good indicator of the next event to occur in the sequence -> To predict the next event in a sequence of event · Allows to compute the probability of the next item in a sequence · Or the probability of a complete sequence · Applications of Language Model..

Bag of Word Model - The order is ignored (in the sentence) - Fast/simple (ex) Multinomial Naïve Bayes text classification(spam filtering) Information Retrieval (google search) - Representation of a documents => Vectors of pairs => Word : all words in the vocabulary (aka term) => Value: a number associated with the word in the document - Different possible schemes (1) Boo..