Clipart-126-DomainNet

Clipart-126-DomainNet is an image classification vision-language encoder model fine-tuned from google/siglip2-base-patch16-224 for a single-label classification task. It is designed to classify clipart images into 126 domain categories using the SiglipForImageClassification architecture.

Moment Matching for Multi-Source Domain Adaptation : https://arxiv.org/pdf/1812.01754

SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features https://arxiv.org/pdf/2502.14786

Classification Report:
                         precision    recall  f1-score   support

       aircraft_carrier     0.8667    0.4643    0.6047        56
            alarm_clock     0.9706    0.8919    0.9296        74
                    ant     0.8889    0.8615    0.8750        65
                  anvil     0.5984    0.6083    0.6033       120
              asparagus     0.8158    0.6078    0.6966        51
                    axe     0.7544    0.5309    0.6232        81
                 banana     0.7111    0.5517    0.6214        58
                 basket     0.8571    0.8182    0.8372        66
                bathtub     0.7531    0.7821    0.7673        78
                   bear     0.9118    0.6458    0.7561        48
                    bee     0.9636    0.9636    0.9636       165
                   bird     0.8967    0.9529    0.9240       255
             blackberry     0.8082    0.8429    0.8252        70
              blueberry     0.8661    0.8981    0.8818       108
              bottlecap     0.7821    0.8299    0.8053       147
               broccoli     0.8947    0.8947    0.8947        95
                    bus     0.9663    0.9348    0.9503        92
              butterfly     0.9333    0.9545    0.9438       132
                 cactus     0.9677    0.9091    0.9375        99
                   cake     0.8750    0.8099    0.8412       121
             calculator     0.9583    0.5897    0.7302        39
                  camel     0.9391    0.9310    0.9351       116
                 camera     0.8846    0.8679    0.8762        53
                 candle     0.8298    0.8478    0.8387        92
                 cannon     0.8551    0.8551    0.8551        69
                  canoe     0.8462    0.7432    0.7914        74
                 carrot     0.8800    0.7719    0.8224        57
                 castle     1.0000    0.8511    0.9195        47
                    cat     0.8167    0.7903    0.8033        62
            ceiling_fan     1.0000    0.2000    0.3333        30
             cell_phone     0.7400    0.6491    0.6916        57
                  cello     0.8372    0.9114    0.8727        79
                  chair     0.8986    0.8378    0.8671        74
             chandelier     0.9617    0.9263    0.9437       190
             coffee_cup     0.8811    0.9389    0.9091       229
                compass     0.9799    0.9012    0.9389       162
               computer     0.7124    0.9045    0.7970       178
                    cow     0.9517    0.9718    0.9617       142
                   crab     0.8738    0.9000    0.8867       100
              crocodile     0.9778    0.9167    0.9462       144
            cruise_ship     0.8544    0.9072    0.8800       194
                    dog     0.8125    0.7761    0.7939        67
                dolphin     0.7680    0.7500    0.7589       128
                 dragon     0.9512    0.9176    0.9341        85
                  drums     0.8919    0.9635    0.9263       137
                   duck     0.8774    0.8447    0.8608       161
               dumbbell     0.9048    0.9500    0.9268       280
               elephant     0.9038    0.8952    0.8995       105
             eyeglasses     0.8636    0.8488    0.8562       291
                feather     0.8564    0.9227    0.8883       181
                  fence     0.9211    0.8400    0.8787       125
                   fish     0.8963    0.8768    0.8864       138
               flamingo     0.9636    0.9381    0.9507       226
                 flower     0.9146    0.9454    0.9298       238
                   foot     0.8780    0.8889    0.8834        81
                   fork     0.9032    0.9091    0.9061       154
                   frog     0.9420    0.9489    0.9455       137
                giraffe     0.9643    0.9153    0.9391       118
                 goatee     0.8763    0.9422    0.9081       173
                 grapes     0.9114    0.8571    0.8834        84
                 guitar     0.9595    0.8554    0.9045        83
                 hammer     0.6111    0.7719    0.6822       114
             helicopter     0.9444    0.9533    0.9488       107
                 helmet     0.7368    0.8550    0.7915       131
                  horse     0.9588    0.9819    0.9702       166
               kangaroo     0.9125    0.8488    0.8795        86
                lantern     0.8254    0.7536    0.7879        69
                 laptop     0.8108    0.5000    0.6186        60
                   leaf     0.7143    0.3333    0.4545        30
                   lion     0.9744    0.8085    0.8837        47
               lipstick     0.7875    0.6632    0.7200        95
                lobster     0.8963    0.9130    0.9046       161
             microphone     0.7925    0.9231    0.8528        91
                 monkey     0.9623    0.9027    0.9315       113
               mosquito     0.8636    0.8444    0.8539        45
                  mouse     0.9167    0.8333    0.8730        66
                    mug     0.8989    0.8163    0.8556        98
               mushroom     0.9429    0.9429    0.9429       105
                  onion     0.9365    0.8429    0.8872       140
                  panda     1.0000    0.9726    0.9861        73
                 peanut     0.5900    0.7195    0.6484        82
                   pear     0.7692    0.7246    0.7463        69
                   peas     0.8000    0.7429    0.7704        70
                 pencil     0.6667    0.0909    0.1600        44
                penguin     0.9717    0.9279    0.9493       111
                    pig     0.9551    0.8252    0.8854       103
                 pillow     0.6290    0.5571    0.5909        70
              pineapple     0.9846    0.8889    0.9343        72
                 potato     0.6038    0.6531    0.6275        98
           power_outlet     0.8636    0.4043    0.5507        47
                  purse     0.0000    0.0000    0.0000        27
                 rabbit     0.9341    0.8586    0.8947        99
                raccoon     0.8836    0.9021    0.8927       143
             rhinoceros     0.8750    0.9459    0.9091        74
                  rifle     0.7595    0.7500    0.7547        80
              saxophone     0.9454    0.9886    0.9665       175
            screwdriver     0.7521    0.6929    0.7213       127
             sea_turtle     0.9677    0.9626    0.9651       187
                see_saw     0.6679    0.8698    0.7556       215
                  sheep     0.9355    0.9158    0.9255        95
                   shoe     0.8969    0.8700    0.8832       100
             skateboard     0.8632    0.8673    0.8652       211
                  snake     0.9302    0.9160    0.9231       131
              speedboat     0.8187    0.8976    0.8563       166
                 spider     0.9043    0.9286    0.9163       112
               squirrel     0.7945    0.8855    0.8375       131
             strawberry     0.8687    0.9923    0.9264       260
            streetlight     0.8178    0.9293    0.8700       198
            string_bean     0.8525    0.8000    0.8254        65
              submarine     0.8022    0.8902    0.8439       164
                   swan     0.8397    0.9003    0.8690       291
                  table     0.8564    0.9200    0.8871       175
                 teapot     0.8763    0.9189    0.8971       185
             teddy-bear     0.9006    0.8953    0.8980       172
             television     0.8509    0.8220    0.8362       118
       the_Eiffel_Tower     0.9468    0.9082    0.9271        98
the_Great_Wall_of_China     0.9462    0.9462    0.9462        93
                  tiger     0.9417    0.9826    0.9617       230
                    toe     0.8250    0.6600    0.7333        50
                  train     0.9362    0.9778    0.9565        90
                  truck     0.9367    0.8916    0.9136        83
               umbrella     0.9633    0.9545    0.9589       110
                   vase     0.7642    0.8393    0.8000       112
             watermelon     0.9527    0.9527    0.9527       148
                  whale     0.7453    0.8144    0.7783       194
                  zebra     0.9275    0.9676    0.9471       185

               accuracy                         0.8691     14818
              macro avg     0.8613    0.8251    0.8351     14818
           weighted avg     0.8705    0.8691    0.8661     14818

The model categorizes images into the following 126 classes:

• Class 0: "aircraft_carrier"
• Class 1: "alarm_clock"
• Class 2: "ant"
• Class 3: "anvil"
• Class 4: "asparagus"
• Class 5: "axe"
• Class 6: "banana"
• Class 7: "basket"
• Class 8: "bathtub"
• Class 9: "bear"
• Class 10: "bee"
• Class 11: "bird"
• Class 12: "blackberry"
• Class 13: "blueberry"
• Class 14: "bottlecap"
• Class 15: "broccoli"
• Class 16: "bus"
• Class 17: "butterfly"
• Class 18: "cactus"
• Class 19: "cake"
• Class 20: "calculator"
• Class 21: "camel"
• Class 22: "camera"
• Class 23: "candle"
• Class 24: "cannon"
• Class 25: "canoe"
• Class 26: "carrot"
• Class 27: "castle"
• Class 28: "cat"
• Class 29: "ceiling_fan"
• Class 30: "cell_phone"
• Class 31: "cello"
• Class 32: "chair"
• Class 33: "chandelier"
• Class 34: "coffee_cup"
• Class 35: "compass"
• Class 36: "computer"
• Class 37: "cow"
• Class 38: "crab"
• Class 39: "crocodile"
• Class 40: "cruise_ship"
• Class 41: "dog"
• Class 42: "dolphin"
• Class 43: "dragon"
• Class 44: "drums"
• Class 45: "duck"
• Class 46: "dumbbell"
• Class 47: "elephant"
• Class 48: "eyeglasses"
• Class 49: "feather"
• Class 50: "fence"
• Class 51: "fish"
• Class 52: "flamingo"
• Class 53: "flower"
• Class 54: "foot"
• Class 55: "fork"
• Class 56: "frog"
• Class 57: "giraffe"
• Class 58: "goatee"
• Class 59: "grapes"
• Class 60: "guitar"
• Class 61: "hammer"
• Class 62: "helicopter"
• Class 63: "helmet"
• Class 64: "horse"
• Class 65: "kangaroo"
• Class 66: "lantern"
• Class 67: "laptop"
• Class 68: "leaf"
• Class 69: "lion"
• Class 70: "lipstick"
• Class 71: "lobster"
• Class 72: "microphone"
• Class 73: "monkey"
• Class 74: "mosquito"
• Class 75: "mouse"
• Class 76: "mug"
• Class 77: "mushroom"
• Class 78: "onion"
• Class 79: "panda"
• Class 80: "peanut"
• Class 81: "pear"
• Class 82: "peas"
• Class 83: "pencil"
• Class 84: "penguin"
• Class 85: "pig"
• Class 86: "pillow"
• Class 87: "pineapple"
• Class 88: "potato"
• Class 89: "power_outlet"
• Class 90: "purse"
• Class 91: "rabbit"
• Class 92: "raccoon"
• Class 93: "rhinoceros"
• Class 94: "rifle"
• Class 95: "saxophone"
• Class 96: "screwdriver"
• Class 97: "sea_turtle"
• Class 98: "see_saw"
• Class 99: "sheep"
• Class 100: "shoe"
• Class 101: "skateboard"
• Class 102: "snake"
• Class 103: "speedboat"
• Class 104: "spider"
• Class 105: "squirrel"
• Class 106: "strawberry"
• Class 107: "streetlight"
• Class 108: "string_bean"
• Class 109: "submarine"
• Class 110: "swan"
• Class 111: "table"
• Class 112: "teapot"
• Class 113: "teddy-bear"
• Class 114: "television"
• Class 115: "the_Eiffel_Tower"
• Class 116: "the_Great_Wall_of_China"
• Class 117: "tiger"
• Class 118: "toe"
• Class 119: "train"
• Class 120: "truck"
• Class 121: "umbrella"
• Class 122: "vase"
• Class 123: "watermelon"
• Class 124: "whale"
• Class 125: "zebra"

Run with Transformers🤗

!pip install -q transformers torch pillow gradio

import gradio as gr
from transformers import AutoImageProcessor, SiglipForImageClassification
from transformers.image_utils import load_image
from PIL import Image
import torch

# Load model and processor
model_name = "prithivMLmods/Clipart-126-DomainNet"
model = SiglipForImageClassification.from_pretrained(model_name)
processor = AutoImageProcessor.from_pretrained(model_name)

def clipart_classification(image):
    """Predicts the clipart category for an input image."""
    # Convert the input numpy array to a PIL Image and ensure it's in RGB format
    image = Image.fromarray(image).convert("RGB")
    
    # Process the image and prepare it for the model
    inputs = processor(images=image, return_tensors="pt")
    
    # Perform inference without gradient computation
    with torch.no_grad():
        outputs = model(**inputs)
        logits = outputs.logits
        # Apply softmax to obtain probabilities for each class
        probs = torch.nn.functional.softmax(logits, dim=1).squeeze().tolist()
    
    # Mapping from indices to clipart category labels
    labels = {
        "0": "aircraft_carrier", "1": "alarm_clock", "2": "ant", "3": "anvil", "4": "asparagus",
        "5": "axe", "6": "banana", "7": "basket", "8": "bathtub", "9": "bear",
        "10": "bee", "11": "bird", "12": "blackberry", "13": "blueberry", "14": "bottlecap",
        "15": "broccoli", "16": "bus", "17": "butterfly", "18": "cactus", "19": "cake",
        "20": "calculator", "21": "camel", "22": "camera", "23": "candle", "24": "cannon",
        "25": "canoe", "26": "carrot", "27": "castle", "28": "cat", "29": "ceiling_fan",
        "30": "cell_phone", "31": "cello", "32": "chair", "33": "chandelier", "34": "coffee_cup",
        "35": "compass", "36": "computer", "37": "cow", "38": "crab", "39": "crocodile",
        "40": "cruise_ship", "41": "dog", "42": "dolphin", "43": "dragon", "44": "drums",
        "45": "duck", "46": "dumbbell", "47": "elephant", "48": "eyeglasses", "49": "feather",
        "50": "fence", "51": "fish", "52": "flamingo", "53": "flower", "54": "foot",
        "55": "fork", "56": "frog", "57": "giraffe", "58": "goatee", "59": "grapes",
        "60": "guitar", "61": "hammer", "62": "helicopter", "63": "helmet", "64": "horse",
        "65": "kangaroo", "66": "lantern", "67": "laptop", "68": "leaf", "69": "lion",
        "70": "lipstick", "71": "lobster", "72": "microphone", "73": "monkey", "74": "mosquito",
        "75": "mouse", "76": "mug", "77": "mushroom", "78": "onion", "79": "panda",
        "80": "peanut", "81": "pear", "82": "peas", "83": "pencil", "84": "penguin",
        "85": "pig", "86": "pillow", "87": "pineapple", "88": "potato", "89": "power_outlet",
        "90": "purse", "91": "rabbit", "92": "raccoon", "93": "rhinoceros", "94": "rifle",
        "95": "saxophone", "96": "screwdriver", "97": "sea_turtle", "98": "see_saw", "99": "sheep",
        "100": "shoe", "101": "skateboard", "102": "snake", "103": "speedboat", "104": "spider",
        "105": "squirrel", "106": "strawberry", "107": "streetlight", "108": "string_bean",
        "109": "submarine", "110": "swan", "111": "table", "112": "teapot", "113": "teddy-bear",
        "114": "television", "115": "the_Eiffel_Tower", "116": "the_Great_Wall_of_China",
        "117": "tiger", "118": "toe", "119": "train", "120": "truck", "121": "umbrella",
        "122": "vase", "123": "watermelon", "124": "whale", "125": "zebra"
    }
    
    # Create a dictionary mapping each label to its corresponding probability (rounded)
    predictions = {labels[str(i)]: round(probs[i], 3) for i in range(len(probs))}
    return predictions

# Create Gradio interface
iface = gr.Interface(
    fn=clipart_classification,
    inputs=gr.Image(type="numpy"),
    outputs=gr.Label(label="Prediction Scores"),
    title="Clipart-126-DomainNet Classification",
    description="Upload a clipart image to classify it into one of 126 domain categories."
)

# Launch the app
if __name__ == "__main__":
    iface.launch()

---

Intended Use:

The Clipart-126-DomainNet model is designed for clipart image classification. It categorizes clipart images into a wide range of domains—from objects like an "aircraft_carrier" or "alarm_clock" to various everyday items. Potential use cases include:

• Digital Art and Design: Assisting designers in organizing and retrieving clipart assets.
• Content Management: Enhancing digital asset management systems with robust clipart classification.
• Creative Search Engines: Enabling clipart-based search for design inspiration and resource curation.
• Computer Vision Research: Serving as a benchmark for studies in clipart recognition and domain adaptation.