UI Design Suggestion of ExplainaBoard Client

[pfliu-nlp]

Current Implementation (in the latest PRs)

from explainaboard_client.client import ExplainaboardClient
from explainaboard_client.config import Config

client = ExplainaboardClient(Config(
    "user_email",
    "api_key",
    "environment"
))

result= client.evaluate_system_file(
    system_output_file="sst2-output.txt",
    system_output_file_type="text",
    task="text-classification",
    system_name="test_cli",
    metric_names=["Accuracy"],
    source_language="en",
    target_language="en",
    dataset="sst2",
    split="test",
    sh

Unfriendly Points

• (1) explainaboard_client.client: too long and verbose and no one wants to remember this
• (2) from explainaboard_client.config import Config: the additional burden to remember this (importing Config)
• (3) users need to import two lines of unfamiliar packages, which are not good
• (4) Config requires too much information, for example, could we remove the need of email or even remove the config!

Some potential better designs in my mind (from worse -> better)

1.

import os
from explainaboard_client import ExplainaboardClient


explainaboard_client.api_key = os.getenv("API_KEY") # api_key could be a global/system variable


result= ExplainaboardClient.evaluate_system_file(
    system_output_file="sst2-output.txt",
    system_output_file_type="text",
    task="text-classification",
    system_name="test_cli",
    metric_names=["Accuracy"],
    source_language="en",
    target_language="en",
    dataset="sst2",
    split="test",
    ..)

2.

import os
from explainaboard_client

explainaboard_client.api_key = os.getenv("API_KEY") 

result= explainaboard_client.ExplainaboardClient.evaluate_system_file(...)

3.

import os
from inspiredco import ExplainaboardClient  # from  inspiredco import EaasClient in the future

inspiredco.api_key = os.getenv("API_KEY")


result= ExplainaboardClient.evaluate_system_file(...)

3.14

import os
from inspiredco

inspiredco.api_key = os.getenv("API_KEY")

result= inspiredco.ExplainaboardClient.evaluate_system_file(...)

[neubig]

Thanks @pfliu-nlp! I think that all the comments here are valid.

I think we should definitely remove Config, I'll do that in a follow-up PR.

Regarding the other design decisions, I think it's reasonably easy to separate the discussion of which functions should be provided (the focus of the current PRs) and how they should be provided (the focus of your discussion here). So as a way to move forward I'd suggest that we review/merge the current PRs, and then after that decide about the design decisions in this issue. Does that sound reasonable?

[pfliu-nlp]

So as a way to move forward I'd suggest that we review/merge the current PRs, and then after that decide about the design decisions in this issue. Does that sound reasonable?

Sounds good!

[pfliu-nlp]

Also the time is ripe for us to have the following feature simultaneously in explainaboard web (displaying code block),