BSNS with Parameter-efficient Fine-tuning via Adapters

In the BSNS framework, each node, tasked with a segment of the LLM, incorporates adapter modules designed to fine-tune the model for specific tasks with minimal computational overhead. The adapters are inserted between the transformer layers, allowing for targeted modifications of the model's behavior without retraining the entire network. The operational dynamics of integrating adapters into BSNS can be outlined as follows.

Operational framework for adapters in Nesa

Adapters are small neural network modules inserted between the layers of a pre-trained model. They allow for task-specific training with minimal parameter updates. Formally, the operation of an adapter within a transformer layer can be represented as:

$$h'_i = \text{LayerNorm}(h_i + W_2 \phi(W_1 h_i)),$$

where (h_i) is the input to the adapter at layer (i), (W_1) and (W_2) are trainable weights of the adapter, (\phi) denotes a non-linear activation function, and (h'_i) is the output of the adapter. LoRA introduces trainable low-rank matrices to adjust the attention and feed-forward networks within transformer layers, offering another parameter-efficient fine-tuning method.

The adaptation of a weight matrix (W) in a transformer can be modeled as:

$$W' = W + BA,$$

where (W) is the original weight matrix of the model, (A \in \mathbb{R}^{r \times n}) and (B \in \mathbb{R}^{n \times r}) are the low-rank matrices introduced by LoRA with (r \ll n), and (W') is the adapted weight matrix. Upon initiating a fine-tuning session, each participating node initializes adapter modules according to the specified configuration, aligning with the shard of the LLM it is responsible for. This initialization includes setting up the adapter architecture (e.g., feed-forward layers within the adapter) and integrating it seamlessly with the existing transformer layers.

Node synchronization

Nodes engage in a collaborative fine-tuning process, where each node updates its respective adapter modules based on the gradients computed from the task-specific data processed through its shard of the LLM. This process involves three steps:

Forward pass. Computing the forward pass through the transformer layers and adapters, generating predictions based on the input data.

Backward pass. Computing gradients based on the loss between predictions and true labels, where gradients relevant to the adapter modules are computed and used for updating the adapter parameters.

Synchronization and update. After computing the updates for the adapters, nodes synchronize these updates across the network to ensure consistency and convergence of the model. This synchronization step is crucial to maintain the integrity of the fine-tuned model across the distributed environment, ensuring that each node's adapters evolve cohesively towards the fine-tuning objective.

The updates to the adapter parameters within each node can be mathematically formulated as follows. Let (\theta_i) represent the parameters of the adapter module in the (i)-th node. The update rule for (\theta_i) can be expressed as:

$$\theta_{i}^{(t+1)} = \theta_{i}^{(t)} - \eta \nabla_{\theta_i} L(\theta_{i}^{(t)}),$$

where (t) denotes the current fine-tuning iteration, (\eta) is the learning rate, and (L) represents the loss function computed based on the output of the node's segment of the LLM and the corresponding ground truth. This update rule ensures that each node independently optimizes its adapter modules while contributing to the global fine-tuning objective.

Nesa's BSNS enhances the efficiency of this distributed fine-tuning process through effective node coordination, leveraging blockchain technology for secure and transparent operation management. Nodes participate in a consensus mechanism to agree on the fine-tuning objectives, data distribution strategies, and synchronization intervals for adapter updates, ensuring a coordinated approach towards model improvement. The BSNS framework facilitates the sharing and reuse of fine-tuned adapters across the network, enabling nodes to leverage pre-existing adaptations for new tasks or further refine them for enhanced performance. This collaborative dynamic allows a community-driven approach to LLM customization and improvement, maximizing resource utilization and accelerating innovation within the ecosystem of distributed LLM applications.