Saturday, February 21, 2015

The Farm in Voxel Farm

In the past months we have been working on a very cool system. I will describe it briefly for now, but I would really like to hear your opinions on this. We call it "The Farm".

The idea is simple: we are building a cloud service where everyone can contribute to any voxel world or scene they like. Whatever you contribute is remembered, but it is also kept on your own layer. Whatever you do, you cannot overwrite other people's content. And it is really up to others to decide whether they see your work.

When they do decide to see your work, your contributions to the scenery will be merged in real time. This is one of the key advantages of using voxels. So it will look as if everything was created as a hole, in reality what you see is the sum of different content streams which remain isolated from each other at all times.

This is pretty much like Twitter (or Facebook). You can tweet anything you want, if you have no followers, nobody will really see it. Like Twitter, this is real time. If someone is following you and you change a part of a world they happen to be looking at, they will see the changes in right away.

What I find powerful about this idea is creative groups can be assembled ad-hoc. If you and a few friends want to create together, there is no need to setup servers or anything like that. People in the group just need to follow each other.

If someone in the group goes rogue, it only takes people to un-follow that person to make all of his/her changes disappear.

We want your grandma using this so we have made it very simple for people to participate. You just need to say you want to use the Farm service for a particular project (a world) and you are set:


This system is also similar to source control systems like GitHub. At some point someone may decide to merge all contributions from a team of builders into a single "stream". Anyone following that stream will get the changes then. A project manager (or a Guild leader) can decide which team member streams to merge, which ones to reject.

We are using a revolutionary server-side architecture to achieve this based on Dynamo/Reactor patterns. We can guarantee very low latency by design, also to remember all your data, but this will likely require a small monthly fee from everyone involved. Commitment to this level of service has really high requirements.

That's it: Everyone participates, everyone decides what to see and it is very simple to get in.

Oh, and very important: You data is always yours.

45 comments:

  1. Boom, this is my brain exploding. I soo want in on any closed alpha test versions ;)

    ReplyDelete
  2. That sounds like a great idea! Can't wait to see what your next big ideas are!
    Keep up the good work.

    ReplyDelete
  3. I felt relieved when you mentioned a small fee. I would waste so much time building virtual castle, bridges, impossible buildings etc if it was free :)

    ReplyDelete
    Replies
    1. You can build all these without a monthly subscription. You would then send your creations to someone who is subscribed and could commit. It will appear under their name, not yours though.

      Delete
    2. Actually they could use your name...

      Delete
    3. I foresee communities forming where lots of people create content, while only one of them has a subscription, and that person then commits it all =P.

      Delete
    4. It sounds like a good idea, but I watch things like Medieval Engineers and I hope for a good game that uses Voxel Farm. A game that would sell millions of copies.

      Delete
    5. Yes that would be nice. Not sure why you would bring up ME. It does not have voxel building or destruction, and I don't think they have sold millions of copies.

      Delete
    6. I don't think ME will sell many copies, but I'm pretty sure that the engine (Vrage) is capable of some voxel building and destruction, at least in Space Engineers it is.

      I don't like that they are getting more publicity than Voxel Farm. There used to be a section about a few games being developed on the old Voxel Farm site. Please tell me they are doing well and we will see some finished games soon.

      Delete
    7. Terrain in ME is voxel, asteroids in SE are voxel too. Those are fairly large voxels. The things you build are polygon prefabs. The destruction is similar to Apex from Havock. The fragments are precomputed. They will be the same whether you use a catapult of a fine chisel.

      These are not voxel systems so there is some confusion out there, but we do not mind at all.

      I do not share your negativity. I love they are getting a lot of publicity. This is not a zero sum game, also you cannot compare the attention you get from announcing a game versus a game-engine. These are very different crowds. In general the fact that people expect more of games now is very good news for us, I am rooting for them.

      Delete
    8. I'll work on my attitude. You are right that them being successful is a good thing. I just don't like the fact that the video "Voxel Farm: Down from the mountain" is 1.5 years old and I still can't take such a walk myself.

      Delete
    9. Soooo... great opportunity to ask how VoxelFarm is doing regarding Structural Integrity. :D

      Congrats on getting picked up by Crowfall! They seem to be closer to having a clear game design than the EQ:N team. I am eager to see how they leverage your code for destruction.

      Delete
    10. Structural integrity, we got it to work. It is missing a lot of polish and we have been swamped lately with a lot of other stuff. But is is coming soon.

      Delete
    11. Very exciting news!

      It's not like voxel representation needs a "killer app", but structural integrity could definitely be one.

      When you say "got it to work" do you mean in a client-side solo app like ME, or have you overcome network limitations and achieved a server-side multi-player implementation?

      Delete
  4. So, would there be one world where everyone works? Or how does it work?

    For example, if I subscribe to two people, will both their things appear in the same world? What if they were working in the same world, but werent subscribed to eachothers stuff, and built things in the same spot?

    I dont think I fully understand the system yet =P.

    ReplyDelete
    Replies
    1. If two people build for the same world, but are not aware of each other, whoever chooses to see both their contributions will merge them on the fly. You will get nonsense wherever they overlap.

      For a given project/world, you would merge data from people you know are working together. The list of people you "follow" can be different from one world to another. Whoever created the world listed the main contributors. That will be the default list you get when you open this world. You can then personalize this list.

      You can remove any of the default contributors and add your friends for instance. If your friends are working together in this world then you will want to merge them. If instead each of your friends is doing a different version of a building in the same spot, you will want to keep only one active at any time.

      The key is following is per world and it is something you can flip very quickly. See the checkboxes in the screenshot, this is how you define what you want to see.

      In short, people will need to communicate to make sure their work blends together and it can be seen as a single stream.

      Delete
    2. Cool! Then another question: Is it possible for a single person to set up multiple streams(?) for other people to follow? For example, in the same world, you make a building for one stream, and then next to it, you make a building on another stream. So that this way, people can sort of choose what creations of the person to see? (As long as the creator put them in different streams or groups)

      Delete
    3. It is possible in principle but you have some requirements to meet. You can use any alias you want for your commits to a world, but you need to own the alias. You own an alias when you are the first one to use it regardless of which world.

      Delete
    4. So you could basically just commit one thing as: "Kamica_House" and then another as "Kamica_Tower" as long as those haven't been used yet in that world, right?

      So its basically like naming objects in an editor for games =).

      Delete
    5. Yes you are correct. But your friends may be tired of having so many Kamicas around. Maybe we should think of something specific just for that.

      Delete
    6. Perhaps an overarching account, in which streams are contained, but I'm sure you guys will find a good solution =P.

      Delete
  5. I really like that idea. Of course, I'd have to play with it to really get what you mean. (Was that subtle enough?).

    Would you also be able to just import 'objects' (I guess a defined chunk of a world) if you were merging?

    ReplyDelete
    Replies
    1. Yes, a merge will let you choose a list of streams (people) to merge into your stream. It would be either restricted to a volume you provide or the entire world. Once the merge is completed the content will be duplicated, your stream for the world will contain a copy.

      Delete
  6. So glad about reading a new entry, really missed it.

    Despite the great concept, the thing with overlapping content from different users is still a nasty issue.
    I think it would help to implement optional warning features, that tells you that someone has created things where you are about to build.

    Also if team streams are not properly handled by users, collaboration might get difficult, for example one does the raw architecture, another one goes decorating and modifying.
    If you want grandmas to play this, it better be most easy and not susceptible to user errors.

    ReplyDelete
    Replies
    1. You are right about this. Any form of source control will be affected by these issues. When people use systems like Git or SVN they still need to agree to some level of coordination, otherwise it becomes very time consuming to deal with the collisions. You must be careful. These systems are full of opportunities for you to shoot yourself on the foot.

      Some planning is required if you want to be productive. Your example would work very well, some can create the large architectural features, others can break down the spaces into rooms, and then other would decorate, paint, etc. That would work if you follow this order.

      If the large feature builder comes after the decoration is done and changes the whole structure, no system can help you there (beyond some warnings, etc). Also if two different people consider they should be doing the same spot of large architecture, it may be a problem when they start deleting each other's work.

      While the system could merge this reasonably well, the actual content would not make much sense. This is not different than merging using Git, nothing in Git guarantees the merged code will compile or do what it is supposed to do.

      Source control will not eliminate the need for planning and organization. It can make make work very easy if you adhere to simple rules, which grandma would have no problem following.

      This quote from Tolstoy comes to mind: "All happy families are alike; each unhappy family is unhappy in its own way." All source control systems feel the same when they work, but when they break they do it in their own hellish ways. I am not sure what kind of hell we will get from voxel-based source control. I rather have a quick visit before making any provisions.

      Delete
    2. If this is all being shared publicly... What if you could select other peoples data streams and add them to your own? This could eliminate potential holes caused by more than one person working on a selection. You would see content creators making pieces while other people gather and seam together larger projects to quickly build very detailed worlds. As a content creator the ultimate satisfaction would come when your work is featured in a popular data stream.

      Delete
    3. We have to test more, but I think that approach works.

      It is currently expensive for us to retrieve a stream. The number of streams you choose to see will at the same time be limited at first. It will be mostly your teammates for the task.

      I do not think this is a problem. Team sizes tend to be small in real life. This is due to human factors. Groups larger than 10 become difficult to coordinate and manage. So instead of growing the teams, you scale up by creating hierarchies.

      Curators are also very important. These are the people who put things together. Imagine there is a map out there that is very popular and to which many people have contributed. A curator could produce a new stream where he has selected what he considers the best and blended it together.

      Delete
  7. Are voxel studio tools as good as those of Landmark or can it be expanded with scripting to be like it?

    ReplyDelete
    Replies
    1. Good/bad is a matter of perception. Be sure you try both :)

      Most of our tools use LSystems, which are a form of scripting.

      With Voxel Studio you can also have binary extensions. If you can do C++ you can create your custom voxel layers. They will be as fast as the other voxel layers in the system.

      Delete
  8. This really does have mind-blowing potential. Hope you keep us all informed about the progress here.

    Two questions:

    1. Would people be allowed to modify each other's data? This would be helpful in collaboration projects, but dangerous in that some noob might come and delete your castle while you sleep ;)

    2. Would importing of custom geometry (obj, fbx...) be possible? Some people would be more comfortable creating objects in regular modeling software, perhaps even painting custom textures (if you allow that sort of thing), or would they be forced to stick to the built-in tools and materials/textures for their creations?

    ReplyDelete
    Replies
    1. 1. Nobody can change your stream, your castle will always be there. However changes may be merged based on timestamps, so if someone added a big box of material Air over your castle in their stream after you were done, his stream will take precedence in the merge. Someone looking at both streams will see air and not a castle.

      2. Yes FBX import is possible. We are currently working on keeping the UV mapping as well, but there are some restrictions. If you add voxels that map to a custom texture, this texture must also be linked into the world project. Otherwise people looking at this world won't have a way to texture the voxels.

      Delete
  9. Can you un-select your own stream? maybe two people did build in the same spot but wanted to see the other version on its own before deciding which one should stay / which one to destroy.

    Instead of destroying can you copy a large area of voxels to another place? I saw that there is a tool to copy voxels but can it support large areas as well (like a tall building)?

    ReplyDelete
    Replies
    1. Yes, currently your name is always added to the streams you want to receive. It would be a matter of UI to allow it to be unchecked.

      Merges could support very large areas. In many cases you will want to merge entire worlds. Merging will be throttled by the system. How soon your merge finishes will be determined by how much content you really need to copy.

      Delete
  10. Miguel, why you don't use some crowdfunding platform as a way of marketing, your product is amazing. And no people enough knows it yet. If you use some popular platforms your project can get a lot more visibility.

    No one asked it, so I should do.
    There is a estimated date for a first release? This fee that you cited is around tenths, hundreds or thousands? If you can't answer this questions please give a hint, I can't wait to know when I'll start to use this, or if I can afford the subscription.

    ReplyDelete
    Replies
    1. As before often said in other Posts he dont want to Crowdfound it. The reason slipped my mind. Read the other blogposts. As for the fee look at voxelfarm.com and for wait for the GDC.

      Delete
  11. When, where and how can I get this in my hands?

    ReplyDelete
    Replies
    1. Wait for the GDC there will be more Infomation distibuted. For now you can get from voxelfarm.com for Tripple A Games look at Pricing Plans.

      Delete
  12. I now have the creator license and I have the studio program on my computer. I've looked through the help files available through the help function in the program, and I understand a lot ... generally ... but not specifically.

    I have looked at the tutorial that results in a two story building shell with rows of 4 arched windows on each side. I tried to reproduce that, but something seems to be missing. I'm assuming that it's the arched window instance file? If the instance file is available in the files given, then I must have missed a step or didn't do the step properly. I also watched the rustic building tutorial, ad it's intriguing for sure! But it went by too quickly for me to gain anything more than a general understanding of how things work.

    I have used one other procedurally generated building software that was eventually discontinued. It used a flow chart style of programming. But I am comfortable with writing it as you did if I just had a bit more instruction.

    As a 3d and virtual world artist with little (but some!) programming experience, I would benefit from a tutorial with assets available that is very detailed so that I don't miss a step. I could help by writing one, if someone is able to walk me through all the steps to create something simple from scratch.

    ReplyDelete
  13. I also bought the creator license. The description said: "Create worlds and build amazing things in them. Work with friends or alone." So I hoped to build minecraft-like in a randomly generated world like shown in the videos. But I guess that is not possible without the required programming skills and assets for trees, rocks etc.? The renderer only shows an empty skybox without any terrain and I was not able to render a simple raw heightmap either.

    ReplyDelete
    Replies
    1. Here you can find three examples: http://docs.voxelfarm.com/example-projects

      We are working on the documentation and adding more example coverage.

      Delete
    2. I tried all three example files. When trying to open, each of them crashes the application.

      Delete
    3. I got them to load up, and I started playing with stuff, but I'd like to understand more about how it's created from scratch. I'm a photoshop guy, not a Blender guy, heh...

      Delete
    4. Nice, I didn't realize there was a version available at a reasonable price. It's interesting to just read through the info you have online.

      I'm curious: why do the example project screenshots seem to lack anti-aliasing, and what are all the small white and gray dots? Are these images rendered using a different system than your screenshots and videos in this blog?

      Delete
  14. We could really use some tutorials on how to set up a new project from scratch! Something I could let me grandma read and she could get started creating a World!

    ReplyDelete
  15. Some company will become the "older kids" minecraft. I know that some disapprove of any comparison to minecraft (for a variety of reasons), but the fact remains that there is a huge, growing population of players who became enthralled with minecraft, but eventually moved away (often reluctantly)and feel they have nowhere to go. Why? They want something more, more graphics, more building tools, more props and textures, more capabilities like the ability to code, to make characters do things. Some want to build their own castle or their own town. Others want to collaborate and build bigger things. While a few dream of building an entire game, most have smaller aspirations, but they want the social experience of a big world to explore, and they want to build in that world. There are efforts out there currently (Landmark, Project Spark and recently Microsoft added a coding mod to Minecraft), but so far they all have come up short. Whichever company nails it (creates the total package) will be bigger than minecraft. How do I know? I am a program administrator in education. I build programs for students to engage them in STEM. I spend a lot of time researching, learning about what engages us and how do we sustain it over time. If you want to hear my ideas, contact Galadhlinn in Landmark. :)

    ReplyDelete