llm classifier：用llm(lamini和llama 2)进行数据分类的工具-凯发k8ag旗舰厅

from lamini import laminiclassifier
llm = laminiclassifier()
prompts={  "": "cats are generally more independent and aloof than s, who are often more social and affectionate. cats are also more territorial and may be more aggressive when defending their territory.  cats are self-grooming animals, using their tongues to keep their coats clean and healthy. cats use body language and vocalizations, such as meowing and purring, to communie.",  "": "dogs are more pack-oriented and tend to be more loyal to their human family.  dogs, on the other hand, often require regular grooming from their owners, including brushing and bathing. dogs use body language and barking to convey their messages. dogs are also more responsive to human commands and can be trained to perform a wide range of tasks.",}
llm.prompt_train(prompts)
llm.save("models/my_model.lamini")

llm.predict(["meow"])>> ["cat"]
llm.predict(["meow", "woof"])>> ["cat", "dog"]

llm.predict(["i  milk", "i  bones"])>> ["dog", "cat"] # wrong!

llm = laminiclassifier()
llm.add_data_to_class("cat", "i like milk.")llm.add_data_to_class("dog", ["i like bones"]) # list of examples is valid too
llm.prompt_train(prompts)
llm.predict(["i like milk", "i like bones"])>> ["cat", "dog"] # correct!

# load datallm.load_examples(saved_examples_path="path/to/examples.jsonl")
# print dataprint(llm.get_data())
# save datallm.saved_examples_path = "path/to/examples.jsonl" # overrides default at /tmp/saved_examples.jsonlllm.save_examples()

{"class_name": "cat", "examples": ["i like milk", "meow"]}{"class_name": "dog", "examples": ["woof", "i like bones"]}

llm = laminiclassifier(augmented_example_count=5) # 10 is default

./train.sh

./train.sh --class "cat: cat_prompt" --class "dog: dog_prompt"

./classify.sh 'woof'

{ 'data': 'woof', 'prediction': 'dog', 'probabilities': array([0.37996491, 0.62003509])}

cats are generally more independent and aloof. cats are also more territorial and may be more aggressive when defending their territory.cats are self-grooming animals, using their tongues to keep their coats clean and healthy. cats use body language and vocalizations,such as meowing and purring, to communicate.  an example cat is whiskers, who is a cat who lives in a house with a human.another example cat is furball, who likes to eat food and sleep.  a famous cat is garfield, who is a cat who likes to eat lasagna.

dogs are social animals that live in groups, called packs, in the wild. they are also highly intelligent and trainable.dogs are also known for their loyalty and affection towards their owners. dogs are also known for their ability to learn andperform a variety of tasks, such as herding, hunting, and guarding.  an example dog is snoopy, who is the best friend ofcharlie brown.  another example dog is clifford, who is a big red dog.

./classify.sh --data "i like to sharpen my claws on the furniture." --data "i like to roll in the mud." --data "i like to run any play with a ball." --data "i like to sleep under the bed and purr." --data "my owner is charlie brown." --data "meow, human! i'm famished! where's my food?" --data "purr-fect." --data "hiss! who dared to wake me from my nap? i'll have my revenge... later." --data "i'm so happy to see you! can we go for a walk/play fetch/get treats now?" --data "i'm feeling a little ruff today, can you give me a belly rub to make me feel better?"

{'data': 'i like to sharpen my claws on the furniture.', 'prediction': 'cat', 'probabilities': array([0.55363432, 0.44636568])}{'data': 'i like to roll in the mud.', 'prediction': 'dog', 'probabilities': array([0.4563782, 0.5436218])}{'data': 'i like to run any play with a ball.', 'prediction': 'dog', 'probabilities': array([0.44391914, 0.55608086])}{'data': 'i like to sleep under the bed and purr.', 'prediction': 'cat', 'probabilities': array([0.51146226, 0.48853774])}{'data': 'my owner is charlie brown.', 'prediction': 'dog', 'probabilities': array([0.40052991, 0.59947009])}{'data': "meow, human! i'm famished! where's my food?", 'prediction': 'cat', 'probabilities': array([0.5172964, 0.4827036])}{'data': 'purr-fect.', 'prediction': 'cat', 'probabilities': array([0.50431873, 0.49568127])}{'data': "hiss! who dared to wake me from my nap? i'll have my revenge... "         'later.', 'prediction': 'cat', 'probabilities': array([0.50088163, 0.49911837])}{'data': "i'm so happy to see you! can we go for a walk/play fetch/get treats "         'now?', 'prediction': 'dog', 'probabilities': array([0.42178513, 0.57821487])}{'data': "i'm feeling a little ruff today, can you give me a belly rub to make "         'me feel better?', 'prediction': 'dog', 'probabilities': array([0.46141002, 0.53858998])}

./train.sh --help
usage: train.py [-h] [--class class [class ...]] [--train train [train ...]] [--save save] [-v]
options:  -h, --help            show this help message and exit  --class class [class ...]                        the classes to use for classification, in the format 'class_name:prompt'.  --train train [train ...]                        the training data to use for classification, in the format 'class_name:data'.  --save save           the path to save the model to.  -v, --verbose         whether to print verbose output.

./classify.sh --help
usage: classify.py [-h] [--data data [data ...]] [--load load] [-v] [classify ...]
positional arguments:  classify              the data to classify.
options:  -h, --help            show this help message and exit  --data data [data ...]                        the training data to use for classification, any string.  --load load           the path to load the model from.  -v, --verbose         whether to print verbose output.

from lamini import laminiclassifier
# create a new classifierclassifier = laminiclassifier()

classes = { "some_class" : "some_prompt" }
classifier.prompt_train(classes)

data = ["example 1", "example 2"]classifier.add_data_to_class("some_class", data)
# don't forget to train after adding dataclassifier.prompt_train()

# classify the data - in a list of string(s)prediction = classifier.predict(list_of_strings)
# get the probabilities for each classprobabilities = classifier.predict_proba(list_of_strings)

classifier.save("some_path")

classifier = laminiclassifier.load("some_path")