Update README.md

README.md

@@ -6,87 +6,140 @@ widget:
 tags:
 - music
 datasets:
-- sander-wood/massive_abcnotation_dataset
+- sander-wood/irishman
 ---
 
 # TunesFormer
 
 ## Model description
 
-TunesFormer is a Transformer-based melody generation system trained on 285,449 melodies with musical forms (represented by control codes), where all scores are represented in ABC notation. It was introduced in the paper [TunesFormer: Forming Tunes with Control Codes](https://arxiv.org/abs/2301.02884) by Wu et al. The code is released in [this repository](https://github.com/sander-wood/tunesformer), and the dataset is released in [huggingface](https://huggingface.co/datasets/sander-wood/massive_abcnotation_dataset). 
+TunesFormer is an efficient Transformer-based dual-decoder model specifically designed for the generation of melodies that adhere to user-defined musical forms. It was introduced in the paper [TunesFormer: Forming Tunes with Control Codes](https://arxiv.org/abs/2301.02884) by Wu et al. The code is released in [this repository](https://github.com/sander-wood/tunesformer), and the dataset is released in [huggingface](https://huggingface.co/datasets/sander-wood/irishman). 
 
-By utilizing specific symbols commonly found in ABC notation to indicate section boundaries, TunesFormer can understand and generate melodies with given musical forms based on control codes. The checkpoint released here is TunesFormer-GP (Global Placement), where all the control codes are placed at the beginning of the ABC notation.
+Trained on 214,122 Irish tunes, TunesFormer utilizes techniques including bar patching and control codes. Bar patching reduces sequence length and generation time, while control codes guide TunesFormer in producing melodies that conform to desired musical forms.
 
-This music generation model is available for online use and experience on [TunesFormer: Forming Tunes with Control Codes](https://huggingface.co/spaces/sander-wood/tunesformer). With this online platform, you can freely explore TunesFormer and receive a generated sheet music output from the model.
+This music generation model is available for online use and experience on [huggingface spaces](https://huggingface.co/spaces/sander-wood/tunesformer). With this online platform, you can freely explore TunesFormer and receive a generated sheet music output from the model.
 
-## Intended uses & limitations
+## ABC Notation
 
-You can use this model for melody generation conditioned on musical forms. All scores generated by this model can be written on one stave (for vocal solo or instrumental solo) in standard classical notation, and are in a variety of styles, e.g., blues, classical, folk, jazz, pop, and world music. The generated tunes are in ABC notation, and can be converted to sheet music or audio using [this website](https://ldzhangyx.github.io/abc/), or [this software](https://sourceforge.net/projects/easyabc/).
+ABC notation is an ASCII-based plain text musical notation system that is commonly used for transcribing traditional music and sharing sheet music online. It provides a simple and concise way to represent musical elements such as notes, rhythms, chords, and more.
 
-TunesFormer supports the generation of up to 8 sections, and up to 32 bars per section. In addition, although TunesFormer mostly generates music correctly according to the control codes, due to the random nature of sampling, the musical structure generated by the model occasionally does not match that specified by the control codes when more than 6 sections are generated, or when more than 17 bars are generated for a single section. For more information, please check [our paper](https://arxiv.org/abs/2301.02884).
+For those looking to interact with ABC notation in various ways, there are several tools available:
+
+1. **[Online ABC Player](https://abc.rectanglered.com/):** This web-based tool allows you to input ABC notation and hear the corresponding audio playback. By pasting your ABC code into the player, you can instantly listen to the tune as it would sound when played.
+
+2. **[ABC Sheet Music Editor - EasyABC](https://easyabc.sourceforge.net/):** EasyABC is a user-friendly software application designed for creating, editing, and formatting ABC notation. Its graphical interface enables you to input your ABC code, preview the sheet music, and make adjustments as necessary.
+
+## Dataset
+
+The **Irish Massive ABC Notation ([IrishMAN](https://huggingface.co/datasets/sander-wood/irishman))** dataset includes 216,284 Irish tunes in ABC notation, divided into 99\% (214,122 tunes) for training and 1\% (2,162 tunes) for validation. These tunes were collected from thesession.org and abcnotation.com, both renowned for sharing traditional music. To ensure uniformity in formatting, all tunes were converted to XML and then back to ABC using [scripts](https://wim.vree.org/svgParse/), and fields containing natural language (e.g., titles and lyrics) were removed.
+
+Each tune is automatically annotated with control codes derived from ABC symbols, as described in Section 2.2. These control codes offer insights into the musical forms and structures of each composition.
+
+In the IrishMAN dataset, a [music21](https://web.mit.edu/music21/doc/index.html#)-filtered [subset](https://huggingface.co/datasets/sander-wood/irishman/blob/main/leadsheet.json) includes 34,211 lead sheets, each human-annotated with chord symbols. It is from this very subset that [TunesFormer](https://huggingface.co/sander-wood/tunesformer) developed its capacity to generate melodies with harmonies.
+
+A noteworthy aspect is the copyright status. All tunes in the dataset are in the public domain, ensuring ethical and legal usage for research and creative projects.
+
+## Control codes
+
+Inspired by [CTRL](https://huggingface.co/ctrl), we incorporate control codes into TunesFormer to represent musical forms. These codes, positioned ahead of the ABC notation, enable users to specify the structures of the generated tunes. The following control codes are introduced:
+
+- **S:number of sections**: determines the number of sections in the entire melody. It counts on several symbols that can be used to represent section boundaries: `[|`, `||`, `|]`, `|:`, `::`, and `:|`. In our dataset, the range is 1 to 8 (e.g., `S:1` for a single-section melody, and `S:8` for a melody with eight sections).
+
+- **B:number of bars**: specifies the desired number of bars within a section. It counts on the bar symbol `|`. In our dataset, the range is 1 to 32 (e.g., `B:1` for a one-bar section, and `B:32` for a section with 32 bars).
+
+- **E:edit distance similarity**: controls the similarity level between the current section $c$ and a previous section $p$ in the melody. It is based on the Levenshtein distance $lev(c,p)$ , quantifying the difference between sections for creating variations or contrasts. Mathematically, it can be expressed as:
+  ```
+  eds(c,p) = 1 - lev(c,p) / max(|c|,|p|)
+  ```
+  where $|c|$ and $|p|$ are the string lengths of the two sections. It is discretized into 11 levels, ranging from no match at all to an exact match (e.g., `E:0` for no similarity, and `E:10` for an exact match).
 
 ### How to use
 
 1. Install dependencies for the code released in [this repository](https://github.com/sander-wood/tunesformer):
 ```
-torch                        1.9.1+cu111
+unidecode                    1.3.6
+torch                        1.13.1+cu116
 samplings                    0.1.7
 transformers                 4.18.0
 ```
 
-2. Set the `control_codes` and `prompt` in the script `run_inference.py` for conditional music generation. 
+2. Set the prompt in `prompt.txt` for conditional music generation. 
 ```
-control_codes = "[SECS_3][BARS_4][SIM_6][BARS_4][SIM_10][SIM_6][BARS_4]"
-prompt = """L:1/4
-M:4/4
-K:C
-"C" C C G G |"F" A A"C" G2 |"G" F F"C" E E |"G" D D"C" C2 ||"""
+S:2
+B:9
+E:4
+B:9
+L:1/8
+M:3/4
+K:D
+ de |"D"  """
 ```
-For TunesFormer, the input is a concatenation of `control_codes` and `prompt`. Both `control_codes` and `prompt` are optional. However, if you need to set the prompt, you must set the control codes.
  
-3. Run the script `run_inference.py`. When running a script for the first time, the downloaded files will be cached for future reuse.
+3. Run the script `generate.py`. When running a script for the first time, the downloaded weights will be cached for future reuse.
 
 ```
-python run_inference.py -num_tunes 3 -max_length 1024 -top_p 0.9 -temperature 1.0 -seed 1
+python generate.py -num_tunes 3 -max_patch 128 -top_p 0.8 -top_k 8 -temperature 1.2 -seed 0 -show_control_code True
 ```
 
-4. Enjoy tunes in the folder `output_tunes`! If you want to convert these ABC tunes to sheet music or audio, please refer to `Intended uses & limitations`.
+4. Enjoy tunes in the folder `output_tunes`! If you want to convert these ABC tunes to sheet music or audio, please refer to `ABC Notation`.
 ```
 X:1
-L:1/4
-M:4/4
-K:C
-"C" C C G G |"F" A A"C" G2 |"G" F F"C" E E |"G" D D"C" C2 ||"C" G G"F" A A |"G" G G"C" E2 | 
-"G" F F"C" E E |"G" D D"C" C2 ||"C" C C G G |"F" A A"C" G2 |"G" F F"C" E E |"G" D D"C" C2 |]
+S:2
+B:9
+E:4
+B:9
+L:1/8
+M:3/4
+K:D
+ de |"D" f2 fedB | A2 AF A2 |"G" B2 Bd B2 |"D" A2 AF A2 |"D" f2 fedB | A2 AF A2 |"G" B2 Bd"A" ce |
+"D" d4 :: de |"D" f2 fdfa |"Em" g2 gfed |"A7" cecA ce |"D" fdfafd |"D" f2 fdfa |"Em" g2 gfed |
+"A7" cecA ce |"D" d4 :|
 
 X:2
-L:1/4
-M:4/4
-K:C
-"C" C C G G |"F" A A"C" G2 |"G" F F"C" E E |"G" D D"C" C2 ||"C" E E"F" F F |"C" G G"F" A2 | 
-"G7" F F"C" E E |"G" D D"C" C2 ||"C" C C G G |"F" A A"C" G2 |"G" F F"C" E E |"G" D D"C" C2 |]
+S:2
+B:9
+E:4
+B:9
+L:1/8
+M:3/4
+K:D
+ de |"D" fdcdBd |"D" AdFAdf |"A" gecAce |"D" fdAdde | fdcdBd |"D" AdFAdf |"A" gecAce |"D" d4 :: a2 |
+"G" gfgbd'b |"D" fad'fad' |"A7" c'bagfe |"D" d'afd A2 |"G" gfgbd'b |"D" fad'fad' |"A7" c'bagfe |
+"D" d4 :|
 
 X:3
-L:1/4
-M:4/4
-K:C
-"C" C C G G |"F" A A"C" G2 |"G" F F"C" E E |"G" D D"C" C2 ||"C" G G"F" A A |"C" G G"F" F2 | 
-"C" E E"G" D D |"G" D D"C" C2 ||"C" C C G G |"F" A A"C" G2 |"G" F F"C" E E |"G" D D"C" C2 |]
+S:2
+B:9
+E:4
+B:9
+L:1/8
+M:3/4
+K:D
+ de |"D" f3 e dc |"G" d2 cB AG |"D" F2 A2 d2 |"A7" f4 e2 |"D" f3 e dc |"G" d2 c2 B2 |"A7" A2 B2 c2 |
+"D" d4 :| FG |"D" A3 B A2 |"D" A3 G FG |"D" A2 d2 f2 |"A7" e4 AA |"G" B2 Bc de |"D" f2 d2 A2 |
+"G" Bc d2"A7" c2 |"D" d4 :|
 ```
 
 ### Usage
 ```
+usage: generate.py [-h] [-num_tunes NUM_TUNES] [-max_patch MAX_PATCH]
+                   [-top_p TOP_P] [-top_k TOP_K] [-temperature TEMPERATURE]
+                   [-seed SEED] [-show_control_code SHOW_CONTROL_CODE]
+
 optional arguments:
   -h, --help            show this help message and exit
   -num_tunes NUM_TUNES  the number of independently computed returned tunes
-  -max_length MAX_LENGTH
-                        integer to define the maximum length in tokens of each
+  -max_patch MAX_PATCH  integer to define the maximum length in tokens of each
                         tune
   -top_p TOP_P          float to define the tokens that are within the sample
                         operation of text generation
+  -top_k TOP_K          integer to define the tokens that are within the
+                        sample operation of text generation
   -temperature TEMPERATURE
                         the temperature of the sampling operation
   -seed SEED            seed for randomstate
+  -show_control_code SHOW_CONTROL_CODE
+                        whether to show control code
 ```
 
 ### BibTeX entry and citation info
@@ -97,11 +150,11 @@ optional arguments:
   
   url = {https://arxiv.org/abs/2301.02884},
   
-  author = {Wu, Shangda and Sun, Maosong},
+  author = {Wu, Shangda and Sun, Xiaobing Li, Feng Yu, Maosong},
   
   keywords = {Sound (cs.SD), Audio and Speech Processing (eess.AS), FOS: Computer and information sciences, FOS: Computer and information sciences, FOS: Electrical engineering, electronic engineering, information engineering, FOS: Electrical engineering, electronic engineering, information engineering},
   
-  title = {TunesFormer: Forming Tunes with Control Codes},
+  title = {TunesFormer: Forming Irish Tunes with Control Codes by Bar Patching},
   
   publisher = {arXiv},