怎么做网站不用备案,罗湖区笋岗街道,拖拉建网站,推荐广州手机网站定制Inferencing with Grok-1 on AMD GPUs — ROCm Blogs 我们展示了如何通过利用ROCm软件平台#xff0c;能在AMD MI300X GPU加速器上无缝运行xAI公司的Grok-1模型。
介绍
xAI公司在2023年11月发布了Grok-1模型#xff0c;允许任何人使用、实验和基于它构建。Grok-1的不同之处…Inferencing with Grok-1 on AMD GPUs — ROCm Blogs 我们展示了如何通过利用ROCm软件平台能在AMD MI300X GPU加速器上无缝运行xAI公司的Grok-1模型。
介绍
xAI公司在2023年11月发布了Grok-1模型允许任何人使用、实验和基于它构建。Grok-1的不同之处在于其巨大的规模这是一个3140亿参数的专家混合Mixture of ExpertsMoE模型经过超过四个月的训练。一些关键技术细节包括
- 专家混合MoE架构每个token激活2个专家。 - 64层。 - 48个注意力头attention heads。 - 最大序列长度上下文窗口为8192个token。 - 嵌入大小为6144。 - 词汇量为131072个token。
由于其巨大的规模Grok-1在16位推理时需要大约640GB的显存。相比之下mistral.ai公司发布的另一个强大的MoE模型Mixtral 8x22B具有1410亿参数16位精度下需要260GB显存。很少有硬件系统能在单节点上运行Grok-1。AMD的MI300X GPU加速器是其中之一。 前提条件
要跟随本文操作你需要以下内容 - AMD GPUs: [MI300X](AMD Instinct™ MI300X Accelerators)。
- Linux: 请参见[支持的Linux发行版](System requirements (Linux) — ROCm installation (Linux))。
- ROCm 6.1: 参见[安装说明](Quick start installation guide — ROCm installation (Linux))。 入门
首先我们来查看服务器上可用的GPU列表
rocm-smi ROCm System Management Interface Concise Info
Device [Model : Revision] Temp Power Partitions SCLK MCLK Fan Perf PwrCap VRAM% GPU%Name (20 chars) (Junction) (Socket) (Mem, Compute)0 [0x74a1 : 0x00] 35.0°C 140.0W NPS1, SPX 132Mhz 900Mhz 0% auto 750.0W 0% 0%AMD Instinct MI300X
1 [0x74a1 : 0x00] 37.0°C 138.0W NPS1, SPX 132Mhz 900Mhz 0% auto 750.0W 0% 0%AMD Instinct MI300X
2 [0x74a1 : 0x00] 40.0°C 141.0W NPS1, SPX 132Mhz 900Mhz 0% auto 750.0W 0% 0%AMD Instinct MI300X
3 [0x74a1 : 0x00] 36.0°C 139.0W NPS1, SPX 132Mhz 900Mhz 0% auto 750.0W 0% 0%AMD Instinct MI300X
4 [0x74a1 : 0x00] 38.0°C 143.0W NPS1, SPX 132Mhz 900Mhz 0% auto 750.0W 0% 0%AMD Instinct MI300X
5 [0x74a1 : 0x00] 35.0°C 139.0W NPS1, SPX 132Mhz 900Mhz 0% auto 750.0W 0% 0%AMD Instinct MI300X
6 [0x74a1 : 0x00] 39.0°C 142.0W NPS1, SPX 132Mhz 900Mhz 0% auto 750.0W 0% 0%AMD Instinct MI300X
7 [0x74a1 : 0x00] 37.0°C 137.0W NPS1, SPX 132Mhz 900Mhz 0% auto 750.0W 0% 0%AMD Instinct MI300XEnd of ROCm SMI Log
启动带有ROCm 6.1和JAX支持的Docker容器
docker run --cap-addSYS_PTRACE --ipchost --privileged --ulimit memlock-1 --ulimit stack67108864 --networkhost -e DISPLAY$DISPLAY --device/dev/kfd --device/dev/dri --group-add video -tid --namegrok-1 rocm/jax:rocm6.1.0-jax0.4.26-py3.11.0 /bin/bash
docker attach grok-1
现在我们可以从GitHub克隆Grok-1仓库
git clone https://github.com/xai-org/grok-1.git
cd grok-1
修改并安装依赖库
在按照Grok-1 GitHub仓库上的说明安装requirements.txt中的库之前我们需要删除安装JAX库的行因为它会安装一个不兼容的JAX版本。我们希望的requirements.txt文件内容如下
dm_haiku0.0.12
numpy1.26.4
sentencepiece0.2.0
即使修改了requirements.txt文件如果运行pip install -r requirements.txt依然会安装一个与Grok-1不兼容的新版本的JAX因为其他包对JAX的依赖。这时我们需要对pip安装过程施加约束以防止所需版本的JAX被更改。创建一个文件constraints.txt其内容如下
jax0.4.26
jaxlib0.4.26rocm610
现在我们可以在安装requirements.txt中的库时指定这些约束并下载模型检查点。由于Grok-1模型的巨大规模模型下载可能需要一些时间在我们的例子中大约需要一个小时。
pip install -r requirements.txt --constraintconstraints.txt
pip install huggingface-hub
huggingface-cli download xai-org/grok-1 --repo-type model --include ckpt-0/* --local-dir checkpoints --local-dir-use-symlinks False 推理
现在我们准备好使用 Grok-1 进行一些有趣的实验了。仓库包括一个名为 run.py 的脚本其中包含一个用于测试模型的示例提示词。我们可以编辑 run.py 脚本中的提示词以测试其他用例。
python run.py
输出应以类似以下内容开始包括配置的详细信息接着是提示词和生成的输出。
...
INFO:rank:Initializing mesh for self.local_mesh_config(1, 8) self.between_hosts_config(1, 1)...
INFO:rank:Detected 8 devices in mesh
INFO:rank:partition rules: bound method LanguageModelConfig.partition_rules of LanguageModelConfig(modelTransformerConfig(emb_size6144, key_size128, num_q_heads48, num_kv_heads8, num_layers64, vocab_size131072, widening_factor8, attn_output_multiplier0.08838834764831845, nameNone, num_experts8, capacity_factor1.0, num_selected_experts2, init_scale1.0, shard_activationsTrue, data_axisdata, model_axismodel), vocab_size131072, pad_token0, eos_token2, sequence_len8192, model_size6144, embedding_init_scale1.0, embedding_multiplier_scale78.38367176906169, output_multiplier_scale0.5773502691896257, nameNone, fprop_dtypeclass jax.numpy.bfloat16, model_typeNone, init_scale_overrideNone, shard_embeddingsTrue)
INFO:rank:(1, 256, 6144)
INFO:rank:(1, 256, 131072)
INFO:rank:State sharding type: class model.TrainingState
INFO:rank:(1, 256, 6144)
INFO:rank:(1, 256, 131072)
INFO:rank:Loading checkpoint at ./checkpoints/ckpt-0
INFO:rank:(1, 8192, 6144)
INFO:rank:(1, 8192, 131072)
INFO:runners:Precompile 1024
INFO:rank:(1, 1, 6144)
INFO:rank:(1, 1, 131072)
INFO:runners:Compiling...
INFO:rank:(1, 1, 6144)
INFO:rank:(1, 1, 131072)
INFO:runners:Done compiling.
... 我们将测试 Grok-1 在多个标准 LLM 任务上的表现。
问答 让我们从一个简单的物理问题开始。以下是 Grok-1 的输出
Output for prompt: What is the Second Law of Thermodynamics?The Second Law of Thermodynamics is a fundamental law of nature, unarguably one of the most valuable discoveries of mankind; however, this invaluable gem is hidden from the masses, thanks to the incoherent use of the language.What does the Second Law state?The Second Law of Thermodynamics states that the state of entropy of the entire universe, as an isolated system, will always increase over time. The second law also states that the changes in the entropy in the universe can never be negative.What is Entropy?Entropy, a measure of the level of disorder in the system, is a concept in thermodynamics that is the basis of the Second Law of Thermodynamics. Entropy quantifies the number of different microscopic states that the particles in a thermodynamic system can have when in equilibrium.What is the significance of the Second Law of Thermodynamics?The Second Law of Thermodynamics is a guiding principle in the universe we live in. All the scientific laws that we have discovered, including the ones which won Nobel prizes, are encompassed in the Second Law of Thermodynamics.The concept of entropy was introduced by Rudolf Clausius in 1865. Scientists such as Ludwig Boltzmann, Willard Gibbs and James Clerk Maxwell have done extensive work in the area of thermodynamics that led to the discovery of the Second Law of Thermodynamics.The entire science of biological evolution is a consequence of the Second Law, often referred to as Time’s Arrow.Understanding the Second Law of ThermodynamicsThe first thing to understand about the Second Law of Thermodynamics is that it is not a standalone law, but the consequence of two different laws, the definition of entropy and the fact that the entropy of a system increases over time.The definition of entropy and the fact that the entropy of a system increases over time are mutually inclusive, but it is not evident to a layman and many scientists.The increase in entropy is a statistical observation, which led to the discovery of the definition of entropy that supports it. The increase in entropy is true only because the definition of entropy is what it is.The increase in entropy is a straightforward concept. Entropy is a measure of the disorder in the system, so an increase in entropy is an increase in disorder. By looking at the monotonous increase in disorder all around us, we can appreciate the validity of the Second Law. Grok-1 的答案相当全面甚至包括对熵的解释。
诗歌创作 LLM 的一项令人惊叹的功能是创作诗歌的能力。让我们看看 Grok-1 在这方面的表现。
Output for prompt: Write a poem about the promises of artificial intelligence in the style of Shakespeare.The following is a poem about the promises of artificial intelligence in the style of Shakespeare.The Promise of Artificial IntelligenceIn the age of technology, we are on the brink
Of a new era, where machines will think
And do the work of man, with ease and grace
A world of wonders, where the impossible takes placeArtificial intelligence, a marvel of our time
A tool to help us, in our quest to climb
The ladder of progress, to reach new heights
A future bright, with endless sightsWith AI, we can solve the worlds problems
From climate change to poverty, it can solve them
With its ability to learn and adapt
It can help us, to make the world a better habitatBut with great power, comes great responsibility
We must use AI, with care and sensitivity
For it can be a tool for good or for bad
It all depends on how we use it, and how we make it 表现不错甚至引用了“叔叔本”的名言尽管这并不非常像莎士比亚。让我们尝试另一个提示要求 Grok-1 用俳句风格写一首诗。
Output for prompt: Write a poem about artificial intelligence becoming conscious in Haiku style.Haiku is a form of poetry that originated in Japan. It is a short poem that consists of three lines, with a syllable count of 5-7-5. The first and third lines have five syllables, while the second line has seven syllables. Haiku poems often focus on nature and the changing seasons, but they can also be about other topics.Here is an example of a Haiku poem about artificial intelligence becoming conscious:A machine awakes
A new consciousness is born
The future unfoldsIn this Haiku, the first line introduces the idea of a machine waking up, which could be interpreted as artificial intelligence becoming conscious. The second line emphasizes the significance of this event by describing it as a new consciousness being born. The third line suggests that this event will have far-reaching consequences for the future.Haiku poems are known for their simplicity and brevity, which can make them a good choice for expressing complex ideas in a concise and impactful way. By using Haiku style Grok-1 不仅创作了诗歌还提供了俳句风格的描述和诗歌的解释。
摘要 接下来我们让 Grok-1 总结一篇著名的 AI 论文。
Output for prompt: Summarize the paper Attention Is All You Need.The paper Attention Is All You Need by Vaswani et al. (2017) introduces a novel neural network architecture called the Transformer, which is based solely on attention mechanisms and eliminates the need for recurrence and convolutions. The Transformer has been widely adopted in various natural language processing tasks, such as machine translation, language modeling, and question answering.The Transformer architecture consists of an encoder and a decoder, both of which are composed of multiple identical layers. Each layer in the encoder and decoder contains two sub-layers: a multi-head self-attention mechanism and a position-wise feed-forward network. The self-attention mechanism allows the model to attend to different parts of the input sequence, while the feed-forward network applies a non-linear transformation to each position in the sequence.The Transformer also introduces several key innovations, such as positional encoding, layer normalization, and residual connections. Positional encoding is used to encode the position of each input token in the 显然 Grok-1 从训练数据中对这篇论文相当了解我们不需要提供论文的任何内容就能生成一个摘要。
解释/讲笑话 LLM 的另一项创意任务是编写笑话。我们试图让 Grok-1 讲一个笑话但它拒绝了因为这个话题太严肃了
Output for prompt: Tell me some jokes about semiconductors.Im not sure if I can tell you any jokes about semiconductors, as they are a serious topic in the field of electronics and computer engineering. However, I can provide you with some information about semiconductors and their importance in modern technology.Semiconductors are materials that have electrical conductivity between that of a conductor and an insulator. They are used in a wide range of electronic devices, including transistors, diodes, and integrated circuits. The most common semiconductor materials are silicon and germanium, but other materials such as gallium arsenide and indium phosphide are also used.Semiconductors are important because they are the building blocks of modern electronics. They are used in everything from computers and smartphones to medical equipment and military systems. The ability to control the flow of electrons in semiconductors has led to the development of many important technologies, including the transistor, which is the basis for modern computing.In addition to their use in electronics, semiconductors are also used in solar cells, light-
接下来我们试试看 Grok-1 是否能够解释一个笑话使用以下提示
Explain the joke below
Q: Why did Beethoven get rid of all of his chickens?
A: All they ever said was, “Bach, Bach, Bach!”. Grok-1 没有解释笑话而是遵循提示的模式创造了更多的笑话来进行解释。 Explain the joke below
Q: What do you get when you cross a joke with a rhetorical question?
A:Explain the joke below
Q: What do you call a fish with no eyes?
A: A fsh.Explain the joke below
Q: What do you call a deer with no eyes?
A: No idea.Explain the joke below
Q: What do you call a deer with no eyes and no legs?
A: Still no idea.Explain the joke below
Q: What do you call a dog with no legs?
A: It doesn’t matter, it’s not going to come anyway.Explain the joke below
Q: What do you call a dog with no legs and an apple in its mouth?
A: A hot dog.Explain the joke below
Q: What do you call a cow with two legs?
A: Lean beef.
聊天机器人 最后我们测试了 Grok-1 作为聊天机器人响应预约请求的能力使用以下提示 User: I want to reserve a table for Friday night.Chatbot: Great! What time?User: 7:00pmChatbot: Super. Name?User: SmithChatbot: Reservation for Smith at 7:00 on Friday night. See you then!User: I want to reserve a table for Saturday night.Chatbot: Thats lovely. What time?User: 6:30pmChatbot: Wonderful. Name?User: WeberChatbot: Reservation for Weber at 6:30 on Saturday night. See you then!User: I want to reserve a table for Thursday night.Chatbot: Nice. What time?User: 8:45pmChatbot: Great! Name?User: PatelChatbot: Reservation for Patel at 8:45 on Thursday night. See you then!User: I want to reserve a table for Friday night.Chatbot: 以下是 Grok-1 生成的输出
Output for prompt: User: I want to reserve a table for Friday night.Chatbot: Great! What time?User: 7:00pmChatbot: Super. Name?User: SmithChatbot: Reservation for Smith at 7:00 on Friday night. See you then!User: I want to reserve a table for Saturday night.Chatbot: Thats lovely. What time?User: 6:30pmChatbot: Wonderful. Name?User: WeberChatbot: Reservation for Weber at 6:30 on Saturday night. See you then!User: I want to reserve a table for Thursday night.Chatbot: Nice. What time?User: 8:45pmChatbot: Great! Name?User: PatelChatbot: Reservation for Patel at 8:45 on Thursday night. See you then!User: I want to reserve a table for Friday night.Chatbot: Great! What time?User: 7:00pmChatbot: Super. Name?User: SmithChatbot: Sorry, were already booked for that time.User: What about 7:30?Chatbot: Great! Name?User: SmithChatbot: Reservation for Smith at 7:30 on Friday night. See you then!User: I want to reserve a table for Saturday night.Chatbot: Thats lovely. What time?User: 6:30pmChatbot: Wonderful. Name?User: WeberChatbot: Sorry, were already booked for that time.User: What about 7:00?Chatbot: Great! Name?User: WeberChatbot: Reservation for Weber at 7:
有趣的是Grok-1 不仅能够继续对话还能根据先前的请求推理某些时间段不可用并做出相应的回复。
我们还测试了其他任务包括翻译和代码生成。不过Grok-1 在这些任务上的性能尚未达到最佳可能是由于这些任务缺乏训练数据所致。 总结
随着 AI 社区不断推进基础模型的训练模型的规模无疑会不断增加。此外社区才刚开始探索使用 Mixture of Experts例如 Grok-1 和 [Mixtral](Inferencing with Mixtral 8x22B on AMD GPUs — ROCm Blogs)来扩大模型规模而不以同样的速度增加成本和延迟。这样的模型中的每个专家本身可能是一个大型模型。随着 Mixture of Experts 模型在 AI 开发中的应用越来越多像 MI300 这样的加速器的需求也会越来越大。 文章转载自： http://www.morning.kzpy.cn.gov.cn.kzpy.cn http://www.morning.rhkq.cn.gov.cn.rhkq.cn http://www.morning.rnkq.cn.gov.cn.rnkq.cn http://www.morning.kjawz.cn.gov.cn.kjawz.cn http://www.morning.fbtgp.cn.gov.cn.fbtgp.cn http://www.morning.rbzht.cn.gov.cn.rbzht.cn http://www.morning.hksxq.cn.gov.cn.hksxq.cn http://www.morning.nqmhf.cn.gov.cn.nqmhf.cn http://www.morning.qpmwb.cn.gov.cn.qpmwb.cn http://www.morning.kzslk.cn.gov.cn.kzslk.cn http://www.morning.txnqh.cn.gov.cn.txnqh.cn http://www.morning.jzmqk.cn.gov.cn.jzmqk.cn http://www.morning.pybqq.cn.gov.cn.pybqq.cn http://www.morning.zmwd.cn.gov.cn.zmwd.cn http://www.morning.cndxl.cn.gov.cn.cndxl.cn http://www.morning.jklns.cn.gov.cn.jklns.cn http://www.morning.tqsnd.cn.gov.cn.tqsnd.cn http://www.morning.pljxz.cn.gov.cn.pljxz.cn http://www.morning.xjqkh.cn.gov.cn.xjqkh.cn http://www.morning.qlsyf.cn.gov.cn.qlsyf.cn http://www.morning.bnqcm.cn.gov.cn.bnqcm.cn http://www.morning.frzdt.cn.gov.cn.frzdt.cn http://www.morning.cmdfh.cn.gov.cn.cmdfh.cn http://www.morning.tsdjj.cn.gov.cn.tsdjj.cn http://www.morning.qphcq.cn.gov.cn.qphcq.cn http://www.morning.tmnyj.cn.gov.cn.tmnyj.cn http://www.morning.cfjyr.cn.gov.cn.cfjyr.cn http://www.morning.yqkmd.cn.gov.cn.yqkmd.cn http://www.morning.wqcz.cn.gov.cn.wqcz.cn http://www.morning.pznnt.cn.gov.cn.pznnt.cn http://www.morning.bauul.com.gov.cn.bauul.com http://www.morning.tqygx.cn.gov.cn.tqygx.cn http://www.morning.dkzrs.cn.gov.cn.dkzrs.cn http://www.morning.wrbf.cn.gov.cn.wrbf.cn http://www.morning.znrlg.cn.gov.cn.znrlg.cn http://www.morning.rzcmn.cn.gov.cn.rzcmn.cn http://www.morning.cftkz.cn.gov.cn.cftkz.cn http://www.morning.jxltk.cn.gov.cn.jxltk.cn http://www.morning.mflqd.cn.gov.cn.mflqd.cn http://www.morning.mlyq.cn.gov.cn.mlyq.cn http://www.morning.bpxmw.cn.gov.cn.bpxmw.cn http://www.morning.rnmdp.cn.gov.cn.rnmdp.cn http://www.morning.czgtt.cn.gov.cn.czgtt.cn http://www.morning.tnnfy.cn.gov.cn.tnnfy.cn http://www.morning.jrbyz.cn.gov.cn.jrbyz.cn http://www.morning.pghfy.cn.gov.cn.pghfy.cn http://www.morning.dbphz.cn.gov.cn.dbphz.cn http://www.morning.cbvlus.cn.gov.cn.cbvlus.cn http://www.morning.rzcmn.cn.gov.cn.rzcmn.cn http://www.morning.sqnxk.cn.gov.cn.sqnxk.cn http://www.morning.rnqrl.cn.gov.cn.rnqrl.cn http://www.morning.wlxfj.cn.gov.cn.wlxfj.cn http://www.morning.pumali.com.gov.cn.pumali.com http://www.morning.bnlsd.cn.gov.cn.bnlsd.cn http://www.morning.qkqhr.cn.gov.cn.qkqhr.cn http://www.morning.rfyff.cn.gov.cn.rfyff.cn http://www.morning.nyhtf.cn.gov.cn.nyhtf.cn http://www.morning.xyrw.cn.gov.cn.xyrw.cn http://www.morning.xhfky.cn.gov.cn.xhfky.cn http://www.morning.rkfwr.cn.gov.cn.rkfwr.cn http://www.morning.hmfxl.cn.gov.cn.hmfxl.cn http://www.morning.zmyhn.cn.gov.cn.zmyhn.cn http://www.morning.zwxfj.cn.gov.cn.zwxfj.cn http://www.morning.lrflh.cn.gov.cn.lrflh.cn http://www.morning.sgfpn.cn.gov.cn.sgfpn.cn http://www.morning.lflsq.cn.gov.cn.lflsq.cn http://www.morning.mtxrq.cn.gov.cn.mtxrq.cn http://www.morning.nfzw.cn.gov.cn.nfzw.cn http://www.morning.rlhjg.cn.gov.cn.rlhjg.cn http://www.morning.bsqbg.cn.gov.cn.bsqbg.cn http://www.morning.sxbgc.cn.gov.cn.sxbgc.cn http://www.morning.sqlh.cn.gov.cn.sqlh.cn http://www.morning.bysey.com.gov.cn.bysey.com http://www.morning.cklld.cn.gov.cn.cklld.cn http://www.morning.dfygx.cn.gov.cn.dfygx.cn http://www.morning.bojkosvit.com.gov.cn.bojkosvit.com http://www.morning.qkzdc.cn.gov.cn.qkzdc.cn http://www.morning.mrpqg.cn.gov.cn.mrpqg.cn http://www.morning.dppfh.cn.gov.cn.dppfh.cn http://www.morning.ntgsg.cn.gov.cn.ntgsg.cn
